Note: This is test shard 1 of 8.
[==========] Running 8 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from AdminCliTest
[ RUN ] AdminCliTest.RebalancerReportOnly
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 01:57:55.999531 19631 test_util.cc:268] Using random seed: -293073128
I20240502 01:57:56.005971 19631 ts_itest-base.cc:112] Starting cluster with:
I20240502 01:57:56.006047 19631 ts_itest-base.cc:113] --------------
I20240502 01:57:56.006063 19631 ts_itest-base.cc:114] 5 tablet servers
I20240502 01:57:56.006078 19631 ts_itest-base.cc:115] 3 replicas per TS
I20240502 01:57:56.006091 19631 ts_itest-base.cc:116] --------------
2024-05-02T01:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T01:57:56Z Disabled control of system clock
I20240502 01:57:56.014544 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:39203
--webserver_interface=127.19.43.254
--webserver_port=0
--builtin_ntp_servers=127.19.43.212:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.43.254:39203 with env {}
W20240502 01:57:56.116257 19643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:56.116446 19643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:56.116483 19643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:56.119410 19643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 01:57:56.119458 19643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:56.119477 19643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 01:57:56.119498 19643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 01:57:56.122977 19643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.43.254:39203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:39203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.19.43.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:56.123765 19643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:56.124770 19643 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:56.131189 19651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.131191 19648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.131214 19649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:56.131366 19643 server_base.cc:1034] running on GCE node
I20240502 01:57:56.131903 19643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:56.132719 19643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:56.133852 19643 hybrid_clock.cc:648] HybridClock initialized: now 1714615076133837 us; error 30 us; skew 500 ppm
I20240502 01:57:56.135476 19643 webserver.cc:460] Webserver started at http://127.19.43.254:36001/ using document root <none> and password file <none>
I20240502 01:57:56.135962 19643 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:56.136014 19643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:56.136178 19643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:56.137710 19643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "b9d0c211be94463895c709625273b697"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.138182 19643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "b9d0c211be94463895c709625273b697"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.141171 19643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 01:57:56.143139 19657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.144057 19643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:57:56.144166 19643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
uuid: "b9d0c211be94463895c709625273b697"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.144241 19643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:56.160176 19643 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:56.160737 19643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:56.160880 19643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:56.166904 19643 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.254:39203
I20240502 01:57:56.166909 19709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.254:39203 every 8 connection(s)
I20240502 01:57:56.167768 19643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 01:57:56.169420 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 19643
I20240502 01:57:56.169556 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 01:57:56.170480 19710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:56.175923 19710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697: Bootstrap starting.
I20240502 01:57:56.178181 19710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:56.178916 19710 log.cc:824] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:56.180598 19710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697: No bootstrap required, opened a new log
I20240502 01:57:56.183445 19710 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9d0c211be94463895c709625273b697" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 39203 } }
I20240502 01:57:56.183646 19710 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:56.183710 19710 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9d0c211be94463895c709625273b697, State: Initialized, Role: FOLLOWER
I20240502 01:57:56.184062 19710 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9d0c211be94463895c709625273b697" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 39203 } }
I20240502 01:57:56.184172 19710 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:57:56.184228 19710 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:57:56.184329 19710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:56.184955 19710 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9d0c211be94463895c709625273b697" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 39203 } }
I20240502 01:57:56.185240 19710 leader_election.cc:302] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9d0c211be94463895c709625273b697; no voters:
I20240502 01:57:56.185478 19710 leader_election.cc:288] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:57:56.185570 19715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:57:56.186141 19715 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [term 1 LEADER]: Becoming Leader. State: Replica: b9d0c211be94463895c709625273b697, State: Running, Role: LEADER
I20240502 01:57:56.186403 19710 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 01:57:56.186702 19715 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9d0c211be94463895c709625273b697" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 39203 } }
I20240502 01:57:56.188161 19716 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9d0c211be94463895c709625273b697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9d0c211be94463895c709625273b697" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 39203 } } }
I20240502 01:57:56.188161 19717 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9d0c211be94463895c709625273b697. Latest consensus state: current_term: 1 leader_uuid: "b9d0c211be94463895c709625273b697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9d0c211be94463895c709625273b697" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 39203 } } }
I20240502 01:57:56.188302 19716 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [sys.catalog]: This master's current role is: LEADER
I20240502 01:57:56.188313 19717 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697 [sys.catalog]: This master's current role is: LEADER
W20240502 01:57:56.190958 19730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 01:57:56.191017 19730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 01:57:56.191087 19729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 01:57:56.191745 19729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 01:57:56.196611 19729 catalog_manager.cc:1347] Generated new cluster ID: 415ff9c8a551428486842a9bbe65cbb1
I20240502 01:57:56.196671 19729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 01:57:56.213941 19729 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 01:57:56.214818 19729 catalog_manager.cc:1504] Loading token signing keys...
I20240502 01:57:56.220201 19729 catalog_manager.cc:5857] T 00000000000000000000000000000000 P b9d0c211be94463895c709625273b697: Generated new TSK 0
I20240502 01:57:56.220793 19729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 01:57:56.228506 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.193:0
--local_ip_for_outbound_sockets=127.19.43.193
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--builtin_ntp_servers=127.19.43.212:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20240502 01:57:56.329779 19734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:56.329973 19734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:56.330016 19734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:56.333029 19734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:56.333125 19734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.193
I20240502 01:57:56.336647 19734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--never_fsync=true
--heap_profile_path=/tmp/kudu.19734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.193
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:56.337455 19734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:56.338542 19734 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:56.345685 19742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.345712 19739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.345686 19740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:56.346011 19734 server_base.cc:1034] running on GCE node
I20240502 01:57:56.346362 19734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:56.346874 19734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:56.348060 19734 hybrid_clock.cc:648] HybridClock initialized: now 1714615076348047 us; error 24 us; skew 500 ppm
I20240502 01:57:56.349685 19734 webserver.cc:460] Webserver started at http://127.19.43.193:33047/ using document root <none> and password file <none>
I20240502 01:57:56.350221 19734 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:56.350276 19734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:56.350455 19734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:56.352020 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4d61b05fba7548b49387f8a5ae7bd2d5"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.352448 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4d61b05fba7548b49387f8a5ae7bd2d5"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.355604 19734 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20240502 01:57:56.357594 19749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.358568 19734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:56.358664 19734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4d61b05fba7548b49387f8a5ae7bd2d5"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.358750 19734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:56.387612 19734 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:56.388306 19734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:56.388466 19734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:56.389025 19734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:56.389911 19734 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:56.389956 19734 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.389999 19734 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:56.390044 19734 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.398011 19734 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.193:37921
I20240502 01:57:56.398058 19868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.193:37921 every 8 connection(s)
I20240502 01:57:56.398957 19734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 01:57:56.403829 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 19734
I20240502 01:57:56.403947 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 01:57:56.406069 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.194:0
--local_ip_for_outbound_sockets=127.19.43.194
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--builtin_ntp_servers=127.19.43.212:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 01:57:56.408185 19869 heartbeater.cc:347] Connected to a master server at 127.19.43.254:39203
I20240502 01:57:56.408396 19869 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:56.408914 19869 heartbeater.cc:511] Master 127.19.43.254:39203 requested a full tablet report, sending...
I20240502 01:57:56.410346 19674 ts_manager.cc:194] Registered new tserver with Master: 4d61b05fba7548b49387f8a5ae7bd2d5 (127.19.43.193:37921)
I20240502 01:57:56.411470 19674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.193:38625
W20240502 01:57:56.508962 19872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:56.509163 19872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:56.509217 19872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:56.512140 19872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:56.512251 19872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.194
I20240502 01:57:56.515692 19872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--never_fsync=true
--heap_profile_path=/tmp/kudu.19872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.194
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:56.516471 19872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:56.517491 19872 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:56.524619 19881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.524642 19878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.524654 19879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:56.524758 19872 server_base.cc:1034] running on GCE node
I20240502 01:57:56.525105 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:56.525581 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:56.526729 19872 hybrid_clock.cc:648] HybridClock initialized: now 1714615076526705 us; error 37 us; skew 500 ppm
I20240502 01:57:56.528422 19872 webserver.cc:460] Webserver started at http://127.19.43.194:44515/ using document root <none> and password file <none>
I20240502 01:57:56.528923 19872 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:56.528975 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:56.529148 19872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:56.530758 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "98a75712e7c64bd5a37a668af014a31b"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.531229 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "98a75712e7c64bd5a37a668af014a31b"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.534274 19872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 01:57:56.536317 19887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.537235 19872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:56.537338 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "98a75712e7c64bd5a37a668af014a31b"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.537422 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:56.575227 19872 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:56.575901 19872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:56.576058 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:56.576625 19872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:56.577541 19872 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:56.577589 19872 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.577649 19872 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:56.577678 19872 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.585847 19872 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.194:33843
I20240502 01:57:56.585937 20003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.194:33843 every 8 connection(s)
I20240502 01:57:56.586819 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 01:57:56.591969 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 19872
I20240502 01:57:56.592059 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 01:57:56.594156 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.195:0
--local_ip_for_outbound_sockets=127.19.43.195
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--builtin_ntp_servers=127.19.43.212:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 01:57:56.595382 20004 heartbeater.cc:347] Connected to a master server at 127.19.43.254:39203
I20240502 01:57:56.595609 20004 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:56.596093 20004 heartbeater.cc:511] Master 127.19.43.254:39203 requested a full tablet report, sending...
I20240502 01:57:56.597070 19674 ts_manager.cc:194] Registered new tserver with Master: 98a75712e7c64bd5a37a668af014a31b (127.19.43.194:33843)
I20240502 01:57:56.597793 19674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.194:42903
W20240502 01:57:56.699434 20008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:56.699633 20008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:56.699672 20008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:56.702663 20008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:56.702766 20008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.195
I20240502 01:57:56.706383 20008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--never_fsync=true
--heap_profile_path=/tmp/kudu.20008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.195
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:56.707295 20008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:56.708352 20008 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:56.715899 20016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.715873 20013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:56.716012 20008 server_base.cc:1034] running on GCE node
W20240502 01:57:56.716123 20014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:56.716495 20008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:56.717046 20008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:56.718191 20008 hybrid_clock.cc:648] HybridClock initialized: now 1714615076718179 us; error 31 us; skew 500 ppm
I20240502 01:57:56.719977 20008 webserver.cc:460] Webserver started at http://127.19.43.195:38043/ using document root <none> and password file <none>
I20240502 01:57:56.720535 20008 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:56.720590 20008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:56.720736 20008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:56.722499 20008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "02dc422813b54621b61e5e3a6f19e91c"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.722965 20008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "02dc422813b54621b61e5e3a6f19e91c"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.725970 20008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:57:56.728055 20022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.729034 20008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:56.729143 20008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "02dc422813b54621b61e5e3a6f19e91c"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.729221 20008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:56.750324 20008 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:56.750977 20008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:56.751128 20008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:56.751667 20008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:56.752552 20008 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:56.752599 20008 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.752650 20008 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:56.752677 20008 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.760715 20008 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.195:33573
I20240502 01:57:56.760740 20140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.195:33573 every 8 connection(s)
I20240502 01:57:56.761620 20008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 01:57:56.769552 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20008
I20240502 01:57:56.769645 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 01:57:56.769961 20141 heartbeater.cc:347] Connected to a master server at 127.19.43.254:39203
I20240502 01:57:56.770206 20141 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:56.770810 20141 heartbeater.cc:511] Master 127.19.43.254:39203 requested a full tablet report, sending...
I20240502 01:57:56.771723 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.196:0
--local_ip_for_outbound_sockets=127.19.43.196
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--builtin_ntp_servers=127.19.43.212:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 01:57:56.771838 19674 ts_manager.cc:194] Registered new tserver with Master: 02dc422813b54621b61e5e3a6f19e91c (127.19.43.195:33573)
I20240502 01:57:56.772374 19674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.195:54819
W20240502 01:57:56.874517 20144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:56.874712 20144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:56.874754 20144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:56.877694 20144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:56.877799 20144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.196
I20240502 01:57:56.881255 20144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--never_fsync=true
--heap_profile_path=/tmp/kudu.20144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.196
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:56.882047 20144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:56.883105 20144 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:56.890326 20152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.890305 20150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:56.890372 20149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:56.890612 20144 server_base.cc:1034] running on GCE node
I20240502 01:57:56.890987 20144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:56.891454 20144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:56.892608 20144 hybrid_clock.cc:648] HybridClock initialized: now 1714615076892596 us; error 29 us; skew 500 ppm
I20240502 01:57:56.894186 20144 webserver.cc:460] Webserver started at http://127.19.43.196:41245/ using document root <none> and password file <none>
I20240502 01:57:56.894685 20144 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:56.894742 20144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:56.894901 20144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:56.896466 20144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "0bc269896e1f436380de949dfd4a8e42"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.896898 20144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "0bc269896e1f436380de949dfd4a8e42"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.899895 20144 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:57:56.901794 20158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.902715 20144 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:56.902833 20144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "0bc269896e1f436380de949dfd4a8e42"
format_stamp: "Formatted at 2024-05-02 01:57:56 on dist-test-slave-mnd3"
I20240502 01:57:56.902930 20144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:56.913492 20144 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:56.914062 20144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:56.914196 20144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:56.914664 20144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:56.915468 20144 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:56.915513 20144 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.915551 20144 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:56.915578 20144 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:56.923327 20144 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.196:45397
I20240502 01:57:56.923377 20273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.196:45397 every 8 connection(s)
I20240502 01:57:56.924137 20144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 01:57:56.926206 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20144
I20240502 01:57:56.926299 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 01:57:56.928264 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:0
--local_ip_for_outbound_sockets=127.19.43.197
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--builtin_ntp_servers=127.19.43.212:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 01:57:56.932546 20274 heartbeater.cc:347] Connected to a master server at 127.19.43.254:39203
I20240502 01:57:56.932734 20274 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:56.933238 20274 heartbeater.cc:511] Master 127.19.43.254:39203 requested a full tablet report, sending...
I20240502 01:57:56.934197 19674 ts_manager.cc:194] Registered new tserver with Master: 0bc269896e1f436380de949dfd4a8e42 (127.19.43.196:45397)
I20240502 01:57:56.934718 19674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.196:54963
W20240502 01:57:57.030277 20277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:57.030458 20277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:57.030514 20277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:57.033454 20277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:57.033572 20277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:57:57.037020 20277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:39203
--never_fsync=true
--heap_profile_path=/tmp/kudu.20277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:57.037742 20277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:57.038761 20277 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:57.045719 20285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:57.045730 20283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:57.045719 20282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:57.045985 20277 server_base.cc:1034] running on GCE node
I20240502 01:57:57.046314 20277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:57.046766 20277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:57.047909 20277 hybrid_clock.cc:648] HybridClock initialized: now 1714615077047878 us; error 43 us; skew 500 ppm
I20240502 01:57:57.049593 20277 webserver.cc:460] Webserver started at http://127.19.43.197:41681/ using document root <none> and password file <none>
I20240502 01:57:57.050108 20277 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:57.050168 20277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:57.050338 20277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:57.051808 20277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "a43e9c79cf8f456791adedde464c1092"
format_stamp: "Formatted at 2024-05-02 01:57:57 on dist-test-slave-mnd3"
I20240502 01:57:57.052224 20277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "a43e9c79cf8f456791adedde464c1092"
format_stamp: "Formatted at 2024-05-02 01:57:57 on dist-test-slave-mnd3"
I20240502 01:57:57.055125 20277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 01:57:57.056942 20292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:57.057814 20277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:57.057919 20277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "a43e9c79cf8f456791adedde464c1092"
format_stamp: "Formatted at 2024-05-02 01:57:57 on dist-test-slave-mnd3"
I20240502 01:57:57.057991 20277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:57.089242 20277 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:57.089787 20277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:57.089921 20277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:57.090430 20277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:57.091233 20277 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:57.091277 20277 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:57.091327 20277 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:57.091352 20277 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:57.099129 20277 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:33379
I20240502 01:57:57.099157 20411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:33379 every 8 connection(s)
I20240502 01:57:57.099927 20277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:57:57.103125 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20277
I20240502 01:57:57.103225 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 01:57:57.108397 20412 heartbeater.cc:347] Connected to a master server at 127.19.43.254:39203
I20240502 01:57:57.108587 20412 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:57.109083 20412 heartbeater.cc:511] Master 127.19.43.254:39203 requested a full tablet report, sending...
I20240502 01:57:57.110076 19674 ts_manager.cc:194] Registered new tserver with Master: a43e9c79cf8f456791adedde464c1092 (127.19.43.197:33379)
I20240502 01:57:57.110601 19674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:55275
I20240502 01:57:57.117183 19631 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 01:57:57.127599 19674 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34374:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 01:57:57.144491 20207 tablet_service.cc:1467] Processing CreateTablet for tablet 74462dd7b9094e0b8103bfa5b14bb718 (DEFAULT_TABLE table=TestTable [id=bfc44babd7c5484fad77093fb011da05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:57:57.144495 19936 tablet_service.cc:1467] Processing CreateTablet for tablet 74462dd7b9094e0b8103bfa5b14bb718 (DEFAULT_TABLE table=TestTable [id=bfc44babd7c5484fad77093fb011da05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:57:57.144495 20341 tablet_service.cc:1467] Processing CreateTablet for tablet 74462dd7b9094e0b8103bfa5b14bb718 (DEFAULT_TABLE table=TestTable [id=bfc44babd7c5484fad77093fb011da05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:57:57.145318 20207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74462dd7b9094e0b8103bfa5b14bb718. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:57.145335 20341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74462dd7b9094e0b8103bfa5b14bb718. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:57.145340 19936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74462dd7b9094e0b8103bfa5b14bb718. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:57.150162 20430 tablet_bootstrap.cc:492] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: Bootstrap starting.
I20240502 01:57:57.151415 20434 tablet_bootstrap.cc:492] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: Bootstrap starting.
I20240502 01:57:57.152120 20430 tablet_bootstrap.cc:654] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:57.152844 20430 log.cc:824] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:57.153070 20434 tablet_bootstrap.cc:654] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:57.153635 20434 log.cc:824] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:57.154243 20430 tablet_bootstrap.cc:492] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: No bootstrap required, opened a new log
I20240502 01:57:57.154233 20431 tablet_bootstrap.cc:492] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: Bootstrap starting.
I20240502 01:57:57.154399 20430 ts_tablet_manager.cc:1367] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 01:57:57.155701 20431 tablet_bootstrap.cc:654] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:57.156247 20431 log.cc:824] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:57.157171 20430 raft_consensus.cc:348] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.157342 20430 raft_consensus.cc:374] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:57.157393 20430 raft_consensus.cc:729] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc269896e1f436380de949dfd4a8e42, State: Initialized, Role: FOLLOWER
I20240502 01:57:57.157444 20434 tablet_bootstrap.cc:492] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: No bootstrap required, opened a new log
I20240502 01:57:57.157613 20434 ts_tablet_manager.cc:1367] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240502 01:57:57.157696 20431 tablet_bootstrap.cc:492] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: No bootstrap required, opened a new log
I20240502 01:57:57.157847 20431 ts_tablet_manager.cc:1367] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:57:57.157800 20430 consensus_queue.cc:260] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.158645 20430 ts_tablet_manager.cc:1398] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20240502 01:57:57.158982 20274 heartbeater.cc:503] Master 127.19.43.254:39203 was elected leader, sending a full tablet report...
I20240502 01:57:57.160885 20431 raft_consensus.cc:348] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.161095 20431 raft_consensus.cc:374] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:57.161024 20434 raft_consensus.cc:348] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.161191 20431 raft_consensus.cc:729] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a75712e7c64bd5a37a668af014a31b, State: Initialized, Role: FOLLOWER
I20240502 01:57:57.161235 20434 raft_consensus.cc:374] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:57.161283 20434 raft_consensus.cc:729] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43e9c79cf8f456791adedde464c1092, State: Initialized, Role: FOLLOWER
I20240502 01:57:57.161592 20431 consensus_queue.cc:260] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.161615 20434 consensus_queue.cc:260] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.162389 20431 ts_tablet_manager.cc:1398] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 01:57:57.162410 20434 ts_tablet_manager.cc:1398] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 01:57:57.162739 20412 heartbeater.cc:503] Master 127.19.43.254:39203 was elected leader, sending a full tablet report...
I20240502 01:57:57.163028 20004 heartbeater.cc:503] Master 127.19.43.254:39203 was elected leader, sending a full tablet report...
W20240502 01:57:57.174906 20275 tablet.cc:2369] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:57:57.338079 20005 tablet.cc:2369] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:57:57.350735 20413 tablet.cc:2369] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:57:57.413523 19869 heartbeater.cc:503] Master 127.19.43.254:39203 was elected leader, sending a full tablet report...
I20240502 01:57:57.459771 20438 raft_consensus.cc:482] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:57:57.459969 20438 raft_consensus.cc:504] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.460999 20438 leader_election.cc:288] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98a75712e7c64bd5a37a668af014a31b (127.19.43.194:33843), a43e9c79cf8f456791adedde464c1092 (127.19.43.197:33379)
I20240502 01:57:57.464897 19958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74462dd7b9094e0b8103bfa5b14bb718" candidate_uuid: "0bc269896e1f436380de949dfd4a8e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a75712e7c64bd5a37a668af014a31b" is_pre_election: true
I20240502 01:57:57.464896 20361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74462dd7b9094e0b8103bfa5b14bb718" candidate_uuid: "0bc269896e1f436380de949dfd4a8e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43e9c79cf8f456791adedde464c1092" is_pre_election: true
I20240502 01:57:57.465217 19958 raft_consensus.cc:2454] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bc269896e1f436380de949dfd4a8e42 in term 0.
I20240502 01:57:57.465217 20361 raft_consensus.cc:2454] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bc269896e1f436380de949dfd4a8e42 in term 0.
I20240502 01:57:57.465667 20162 leader_election.cc:302] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc269896e1f436380de949dfd4a8e42, a43e9c79cf8f456791adedde464c1092; no voters:
I20240502 01:57:57.465903 20438 raft_consensus.cc:2781] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:57:57.465978 20438 raft_consensus.cc:482] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:57:57.466008 20438 raft_consensus.cc:3037] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:57.466898 20438 raft_consensus.cc:504] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.467258 20438 leader_election.cc:288] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [CANDIDATE]: Term 1 election: Requested vote from peers 98a75712e7c64bd5a37a668af014a31b (127.19.43.194:33843), a43e9c79cf8f456791adedde464c1092 (127.19.43.197:33379)
I20240502 01:57:57.467603 19958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74462dd7b9094e0b8103bfa5b14bb718" candidate_uuid: "0bc269896e1f436380de949dfd4a8e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a75712e7c64bd5a37a668af014a31b"
I20240502 01:57:57.467624 20361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74462dd7b9094e0b8103bfa5b14bb718" candidate_uuid: "0bc269896e1f436380de949dfd4a8e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43e9c79cf8f456791adedde464c1092"
I20240502 01:57:57.467729 20361 raft_consensus.cc:3037] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:57.467730 19958 raft_consensus.cc:3037] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:57.468758 20361 raft_consensus.cc:2454] T 74462dd7b9094e0b8103bfa5b14bb718 P a43e9c79cf8f456791adedde464c1092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bc269896e1f436380de949dfd4a8e42 in term 1.
I20240502 01:57:57.468758 19958 raft_consensus.cc:2454] T 74462dd7b9094e0b8103bfa5b14bb718 P 98a75712e7c64bd5a37a668af014a31b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bc269896e1f436380de949dfd4a8e42 in term 1.
I20240502 01:57:57.469035 20162 leader_election.cc:302] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc269896e1f436380de949dfd4a8e42, a43e9c79cf8f456791adedde464c1092; no voters:
I20240502 01:57:57.469180 20438 raft_consensus.cc:2781] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:57:57.469389 20438 raft_consensus.cc:686] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [term 1 LEADER]: Becoming Leader. State: Replica: 0bc269896e1f436380de949dfd4a8e42, State: Running, Role: LEADER
I20240502 01:57:57.469677 20438 consensus_queue.cc:237] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } }
I20240502 01:57:57.472635 19674 catalog_manager.cc:5484] T 74462dd7b9094e0b8103bfa5b14bb718 P 0bc269896e1f436380de949dfd4a8e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bc269896e1f436380de949dfd4a8e42 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "0bc269896e1f436380de949dfd4a8e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a75712e7c64bd5a37a668af014a31b" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 33843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bc269896e1f436380de949dfd4a8e42" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 45397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a43e9c79cf8f456791adedde464c1092" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 33379 } health_report { overall_health: UNKNOWN } } }
I20240502 01:57:57.496682 19631 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 01:57:57.498179 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 4d61b05fba7548b49387f8a5ae7bd2d5 to finish bootstrapping
I20240502 01:57:57.509006 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 98a75712e7c64bd5a37a668af014a31b to finish bootstrapping
I20240502 01:57:57.513490 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 02dc422813b54621b61e5e3a6f19e91c to finish bootstrapping
I20240502 01:57:57.518399 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 0bc269896e1f436380de949dfd4a8e42 to finish bootstrapping
I20240502 01:57:57.522711 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver a43e9c79cf8f456791adedde464c1092 to finish bootstrapping
I20240502 01:57:57.697954 20207 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:57:57.698132 19800 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.698813 20073 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.699039 19936 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.700330 20341 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.774204 20141 heartbeater.cc:503] Master 127.19.43.254:39203 was elected leader, sending a full tablet report...
I20240502 01:57:57.782102 20073 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.782619 19936 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.782809 20341 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.787696 19800 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:57.788548 20207 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:57:57.820356 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 19734
I20240502 01:57:57.827556 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 19872
I20240502 01:57:57.833348 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20008
I20240502 01:57:57.838093 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20144
I20240502 01:57:57.842633 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20277
I20240502 01:57:57.847326 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 19643
2024-05-02T01:57:57Z chronyd exiting
[ OK ] AdminCliTest.RebalancerReportOnly (1869 ms)
[----------] 1 test from AdminCliTest (1869 ms total)
[----------] 1 test from RebalanceStartCriteriaTest
[ RUN ] RebalanceStartCriteriaTest.TabletServerIsDown/0
I20240502 01:57:57.867693 19631 test_util.cc:268] Using random seed: -291204950
I20240502 01:57:57.868371 19631 ts_itest-base.cc:112] Starting cluster with:
I20240502 01:57:57.868407 19631 ts_itest-base.cc:113] --------------
I20240502 01:57:57.868422 19631 ts_itest-base.cc:114] 5 tablet servers
I20240502 01:57:57.868433 19631 ts_itest-base.cc:115] 3 replicas per TS
I20240502 01:57:57.868446 19631 ts_itest-base.cc:116] --------------
2024-05-02T01:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T01:57:57Z Disabled control of system clock
I20240502 01:57:57.875267 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:41121
--webserver_interface=127.19.43.254
--webserver_port=0
--builtin_ntp_servers=127.19.43.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.43.254:41121
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 01:57:57.974040 20524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:57.974241 20524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:57.974277 20524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:57.977118 20524 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:57:57.977173 20524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 01:57:57.977196 20524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:57.977217 20524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 01:57:57.977236 20524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 01:57:57.980706 20524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.19.43.254:41121
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:41121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.19.43.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.20524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:57.981475 20524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:57.982683 20524 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:57.989600 20532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:57.989600 20529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:57.989641 20530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:57.989929 20524 server_base.cc:1034] running on GCE node
I20240502 01:57:57.990402 20524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:57.991245 20524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:57.992388 20524 hybrid_clock.cc:648] HybridClock initialized: now 1714615077992376 us; error 26 us; skew 500 ppm
I20240502 01:57:57.994176 20524 webserver.cc:460] Webserver started at http://127.19.43.254:34791/ using document root <none> and password file <none>
I20240502 01:57:57.994699 20524 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:57.994752 20524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:57.994930 20524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:57.996471 20524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c9976c8c34ac4557999b48c4f4e3740d"
format_stamp: "Formatted at 2024-05-02 01:57:57 on dist-test-slave-mnd3"
I20240502 01:57:57.996913 20524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c9976c8c34ac4557999b48c4f4e3740d"
format_stamp: "Formatted at 2024-05-02 01:57:57 on dist-test-slave-mnd3"
I20240502 01:57:57.999940 20524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240502 01:57:58.001859 20538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.002789 20524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:58.002897 20524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c9976c8c34ac4557999b48c4f4e3740d"
format_stamp: "Formatted at 2024-05-02 01:57:57 on dist-test-slave-mnd3"
I20240502 01:57:58.002975 20524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:58.027900 20524 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:58.028530 20524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:58.028674 20524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:58.034869 20524 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.254:41121
I20240502 01:57:58.034879 20592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.254:41121 every 8 connection(s)
I20240502 01:57:58.035784 20524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 01:57:58.038259 20593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:58.039031 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20524
I20240502 01:57:58.039139 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 01:57:58.044193 20593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d: Bootstrap starting.
I20240502 01:57:58.046298 20593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:58.047040 20593 log.cc:824] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:58.048529 20593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d: No bootstrap required, opened a new log
I20240502 01:57:58.050966 20593 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9976c8c34ac4557999b48c4f4e3740d" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 41121 } }
I20240502 01:57:58.051144 20593 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:58.051198 20593 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9976c8c34ac4557999b48c4f4e3740d, State: Initialized, Role: FOLLOWER
I20240502 01:57:58.051575 20593 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9976c8c34ac4557999b48c4f4e3740d" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 41121 } }
I20240502 01:57:58.051687 20593 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:57:58.051751 20593 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:57:58.051851 20593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:58.052597 20593 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9976c8c34ac4557999b48c4f4e3740d" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 41121 } }
I20240502 01:57:58.052906 20593 leader_election.cc:302] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9976c8c34ac4557999b48c4f4e3740d; no voters:
I20240502 01:57:58.053157 20593 leader_election.cc:288] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:57:58.053263 20598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:57:58.053458 20598 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [term 1 LEADER]: Becoming Leader. State: Replica: c9976c8c34ac4557999b48c4f4e3740d, State: Running, Role: LEADER
I20240502 01:57:58.053997 20598 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9976c8c34ac4557999b48c4f4e3740d" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 41121 } }
I20240502 01:57:58.054155 20593 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [sys.catalog]: configured and running, proceeding with master startup.
I20240502 01:57:58.055454 20600 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9976c8c34ac4557999b48c4f4e3740d. Latest consensus state: current_term: 1 leader_uuid: "c9976c8c34ac4557999b48c4f4e3740d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9976c8c34ac4557999b48c4f4e3740d" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 41121 } } }
I20240502 01:57:58.055482 20599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9976c8c34ac4557999b48c4f4e3740d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9976c8c34ac4557999b48c4f4e3740d" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 41121 } } }
I20240502 01:57:58.055624 20599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [sys.catalog]: This master's current role is: LEADER
I20240502 01:57:58.055949 20611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 01:57:58.055871 20600 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d [sys.catalog]: This master's current role is: LEADER
I20240502 01:57:58.058547 20611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 01:57:58.062870 20611 catalog_manager.cc:1347] Generated new cluster ID: 4ec952f841f94073a2ea6e0afe556caa
I20240502 01:57:58.062929 20611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 01:57:58.072337 20611 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 01:57:58.073123 20611 catalog_manager.cc:1504] Loading token signing keys...
I20240502 01:57:58.081640 20611 catalog_manager.cc:5857] T 00000000000000000000000000000000 P c9976c8c34ac4557999b48c4f4e3740d: Generated new TSK 0
I20240502 01:57:58.082266 20611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 01:57:58.092641 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.193:0
--local_ip_for_outbound_sockets=127.19.43.193
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--builtin_ntp_servers=127.19.43.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:57:58.195873 20620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:58.196060 20620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:58.196101 20620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:58.199069 20620 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:57:58.199126 20620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:58.199198 20620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.193
I20240502 01:57:58.202682 20620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.20620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.193
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:58.203480 20620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:58.204512 20620 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:58.211966 20627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.212046 20629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.212062 20625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.212242 20620 server_base.cc:1034] running on GCE node
I20240502 01:57:58.212589 20620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:58.213099 20620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:58.214529 20620 hybrid_clock.cc:648] HybridClock initialized: now 1714615078214514 us; error 26 us; skew 500 ppm
I20240502 01:57:58.216475 20620 webserver.cc:460] Webserver started at http://127.19.43.193:41129/ using document root <none> and password file <none>
I20240502 01:57:58.216995 20620 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:58.217051 20620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:58.217228 20620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:58.218813 20620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "cd4e33ce67ee42eab7d5997131ae5927"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.219252 20620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "cd4e33ce67ee42eab7d5997131ae5927"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.222267 20620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 01:57:58.224215 20635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.225116 20620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:58.225235 20620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "cd4e33ce67ee42eab7d5997131ae5927"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.225309 20620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:58.251540 20620 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:58.252177 20620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:58.252350 20620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:58.252889 20620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:58.253780 20620 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:58.253829 20620 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.253883 20620 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:58.253911 20620 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.261723 20620 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.193:44263
I20240502 01:57:58.261765 20762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.193:44263 every 8 connection(s)
I20240502 01:57:58.262619 20620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 01:57:58.267647 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20620
I20240502 01:57:58.267731 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 01:57:58.269677 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.194:0
--local_ip_for_outbound_sockets=127.19.43.194
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--builtin_ntp_servers=127.19.43.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:57:58.271260 20763 heartbeater.cc:347] Connected to a master server at 127.19.43.254:41121
I20240502 01:57:58.271459 20763 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:58.271926 20763 heartbeater.cc:511] Master 127.19.43.254:41121 requested a full tablet report, sending...
I20240502 01:57:58.273216 20556 ts_manager.cc:194] Registered new tserver with Master: cd4e33ce67ee42eab7d5997131ae5927 (127.19.43.193:44263)
I20240502 01:57:58.274394 20556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.193:33969
W20240502 01:57:58.371102 20768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:58.371317 20768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:58.371366 20768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:58.374351 20768 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:57:58.374408 20768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:58.374498 20768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.194
I20240502 01:57:58.377933 20768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.20768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.194
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:58.378775 20768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:58.379849 20768 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:58.387279 20773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.387290 20776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.387518 20774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.387532 20768 server_base.cc:1034] running on GCE node
I20240502 01:57:58.387925 20768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:58.388453 20768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:58.389600 20768 hybrid_clock.cc:648] HybridClock initialized: now 1714615078389589 us; error 22 us; skew 500 ppm
I20240502 01:57:58.391391 20768 webserver.cc:460] Webserver started at http://127.19.43.194:44301/ using document root <none> and password file <none>
I20240502 01:57:58.391906 20768 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:58.391963 20768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:58.392127 20768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:58.393738 20768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "72a384193f264c0f975debcd152c98cf"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.396546 20768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "72a384193f264c0f975debcd152c98cf"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.399816 20768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:57:58.401821 20783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.402793 20768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:58.402892 20768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "72a384193f264c0f975debcd152c98cf"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.402971 20768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:58.419708 20768 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:58.420217 20768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:58.420361 20768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:58.420825 20768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:58.421648 20768 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:58.421695 20768 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.421746 20768 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:58.421772 20768 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.429358 20768 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.194:39033
I20240502 01:57:58.429397 20901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.194:39033 every 8 connection(s)
I20240502 01:57:58.430256 20768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 01:57:58.435238 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20768
I20240502 01:57:58.435348 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 01:57:58.437533 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.195:0
--local_ip_for_outbound_sockets=127.19.43.195
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--builtin_ntp_servers=127.19.43.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:57:58.438700 20902 heartbeater.cc:347] Connected to a master server at 127.19.43.254:41121
I20240502 01:57:58.438905 20902 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:58.439311 20902 heartbeater.cc:511] Master 127.19.43.254:41121 requested a full tablet report, sending...
I20240502 01:57:58.440192 20556 ts_manager.cc:194] Registered new tserver with Master: 72a384193f264c0f975debcd152c98cf (127.19.43.194:39033)
I20240502 01:57:58.440726 20556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.194:55513
W20240502 01:57:58.538865 20905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:58.539062 20905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:58.539108 20905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:58.542062 20905 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:57:58.542122 20905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:58.542222 20905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.195
I20240502 01:57:58.545660 20905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.20905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.195
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:58.546543 20905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:58.547636 20905 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:58.555084 20912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.555085 20914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.555311 20905 server_base.cc:1034] running on GCE node
W20240502 01:57:58.555353 20911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.555866 20905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:58.556396 20905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:58.557547 20905 hybrid_clock.cc:648] HybridClock initialized: now 1714615078557532 us; error 29 us; skew 500 ppm
I20240502 01:57:58.559286 20905 webserver.cc:460] Webserver started at http://127.19.43.195:35787/ using document root <none> and password file <none>
I20240502 01:57:58.559767 20905 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:58.559815 20905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:58.559963 20905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:58.561491 20905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f2eaeb7384cf4a95be01e4aef273d0dc"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.561882 20905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f2eaeb7384cf4a95be01e4aef273d0dc"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.564872 20905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240502 01:57:58.566833 20920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.567750 20905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20240502 01:57:58.567845 20905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f2eaeb7384cf4a95be01e4aef273d0dc"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.567926 20905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:58.579473 20905 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:58.579941 20905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:58.580068 20905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:58.580502 20905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:58.581283 20905 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:58.581327 20905 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.581363 20905 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:58.581386 20905 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.589114 20905 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.195:36883
I20240502 01:57:58.589141 21037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.195:36883 every 8 connection(s)
I20240502 01:57:58.589948 20905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 01:57:58.592669 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 20905
I20240502 01:57:58.592772 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 01:57:58.594801 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.196:0
--local_ip_for_outbound_sockets=127.19.43.196
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--builtin_ntp_servers=127.19.43.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:57:58.598474 21038 heartbeater.cc:347] Connected to a master server at 127.19.43.254:41121
I20240502 01:57:58.598666 21038 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:58.599046 21038 heartbeater.cc:511] Master 127.19.43.254:41121 requested a full tablet report, sending...
I20240502 01:57:58.599988 20556 ts_manager.cc:194] Registered new tserver with Master: f2eaeb7384cf4a95be01e4aef273d0dc (127.19.43.195:36883)
I20240502 01:57:58.600682 20556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.195:43539
W20240502 01:57:58.696081 21041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:58.696300 21041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:58.696350 21041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:58.699293 21041 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:57:58.699357 21041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:58.699421 21041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.196
I20240502 01:57:58.702893 21041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.21041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.196
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:58.703720 21041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:58.704785 21041 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:58.712417 21046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.712427 21047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.712567 21041 server_base.cc:1034] running on GCE node
W20240502 01:57:58.712739 21049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.713083 21041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:58.713579 21041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:58.714725 21041 hybrid_clock.cc:648] HybridClock initialized: now 1714615078714714 us; error 24 us; skew 500 ppm
I20240502 01:57:58.716500 21041 webserver.cc:460] Webserver started at http://127.19.43.196:37381/ using document root <none> and password file <none>
I20240502 01:57:58.717007 21041 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:58.717059 21041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:58.717236 21041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:58.718849 21041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d389ebfa0fb841c3bca626080ea463f3"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.719312 21041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d389ebfa0fb841c3bca626080ea463f3"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.722337 21041 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 01:57:58.724314 21055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.725214 21041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:57:58.725312 21041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d389ebfa0fb841c3bca626080ea463f3"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.725394 21041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:58.751756 21041 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:58.752378 21041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:58.752538 21041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:58.753082 21041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:58.753985 21041 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:58.754051 21041 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.754107 21041 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:58.754132 21041 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.762099 21041 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.196:38587
I20240502 01:57:58.762113 21172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.196:38587 every 8 connection(s)
I20240502 01:57:58.762989 21041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 01:57:58.769829 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 21041
I20240502 01:57:58.769913 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 01:57:58.771390 21173 heartbeater.cc:347] Connected to a master server at 127.19.43.254:41121
I20240502 01:57:58.771600 21173 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:58.772078 21173 heartbeater.cc:511] Master 127.19.43.254:41121 requested a full tablet report, sending...
I20240502 01:57:58.772318 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:0
--local_ip_for_outbound_sockets=127.19.43.197
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--builtin_ntp_servers=127.19.43.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:57:58.772980 20554 ts_manager.cc:194] Registered new tserver with Master: d389ebfa0fb841c3bca626080ea463f3 (127.19.43.196:38587)
I20240502 01:57:58.773635 20554 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.196:44703
W20240502 01:57:58.874904 21176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:57:58.875103 21176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:57:58.875159 21176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:57:58.878247 21176 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:57:58.878310 21176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:57:58.878396 21176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:57:58.881824 21176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:41121
--never_fsync=true
--heap_profile_path=/tmp/kudu.21176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:57:58.882640 21176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:57:58.883688 21176 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:57:58.891077 21184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.891068 21181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:57:58.891073 21182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:57:58.891222 21176 server_base.cc:1034] running on GCE node
I20240502 01:57:58.891630 21176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:57:58.892130 21176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:57:58.893285 21176 hybrid_clock.cc:648] HybridClock initialized: now 1714615078893270 us; error 28 us; skew 500 ppm
I20240502 01:57:58.895040 21176 webserver.cc:460] Webserver started at http://127.19.43.197:46765/ using document root <none> and password file <none>
I20240502 01:57:58.895545 21176 fs_manager.cc:359] Metadata directory not provided
I20240502 01:57:58.895601 21176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:57:58.895767 21176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:57:58.897363 21176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "01c40b3831064a1aa20aae271625f4cc"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.897806 21176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "01c40b3831064a1aa20aae271625f4cc"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.901095 21176 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 01:57:58.903175 21190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.904078 21176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:57:58.904189 21176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "01c40b3831064a1aa20aae271625f4cc"
format_stamp: "Formatted at 2024-05-02 01:57:58 on dist-test-slave-mnd3"
I20240502 01:57:58.904270 21176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:57:58.933256 21176 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:57:58.933804 21176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:57:58.933956 21176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:57:58.934465 21176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:57:58.935292 21176 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:57:58.935338 21176 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.935380 21176 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:57:58.935408 21176 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:57:58.942998 21176 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:43631
I20240502 01:57:58.943024 21306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:43631 every 8 connection(s)
I20240502 01:57:58.943850 21176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:57:58.947922 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 21176
I20240502 01:57:58.948041 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 01:57:58.951923 21307 heartbeater.cc:347] Connected to a master server at 127.19.43.254:41121
I20240502 01:57:58.952102 21307 heartbeater.cc:464] Registering TS with master...
I20240502 01:57:58.952466 21307 heartbeater.cc:511] Master 127.19.43.254:41121 requested a full tablet report, sending...
I20240502 01:57:58.953274 20554 ts_manager.cc:194] Registered new tserver with Master: 01c40b3831064a1aa20aae271625f4cc (127.19.43.197:43631)
I20240502 01:57:58.953819 20554 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:43329
I20240502 01:57:58.961701 19631 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 01:57:58.969463 20554 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54938:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 01:57:58.985682 20967 tablet_service.cc:1467] Processing CreateTablet for tablet 77955270d5694232af40e471a331e796 (DEFAULT_TABLE table=TestTable [id=8411974974224f54941c9bc1c7b02fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:57:58.985683 20831 tablet_service.cc:1467] Processing CreateTablet for tablet 77955270d5694232af40e471a331e796 (DEFAULT_TABLE table=TestTable [id=8411974974224f54941c9bc1c7b02fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:57:58.986685 20967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77955270d5694232af40e471a331e796. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:58.986685 20831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77955270d5694232af40e471a331e796. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:58.987087 20689 tablet_service.cc:1467] Processing CreateTablet for tablet 77955270d5694232af40e471a331e796 (DEFAULT_TABLE table=TestTable [id=8411974974224f54941c9bc1c7b02fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:57:58.987787 20689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77955270d5694232af40e471a331e796. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:57:58.992259 21324 tablet_bootstrap.cc:492] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: Bootstrap starting.
I20240502 01:57:58.992910 21325 tablet_bootstrap.cc:492] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: Bootstrap starting.
I20240502 01:57:58.992971 21326 tablet_bootstrap.cc:492] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: Bootstrap starting.
I20240502 01:57:58.994688 21326 tablet_bootstrap.cc:654] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:58.994869 21324 tablet_bootstrap.cc:654] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:58.994868 21325 tablet_bootstrap.cc:654] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: Neither blocks nor log segments found. Creating new log.
I20240502 01:57:58.995486 21326 log.cc:824] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:58.995584 21324 log.cc:824] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:58.995584 21325 log.cc:824] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: Log is configured to *not* fsync() on all Append() calls
I20240502 01:57:58.996933 21325 tablet_bootstrap.cc:492] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: No bootstrap required, opened a new log
I20240502 01:57:58.997020 21324 tablet_bootstrap.cc:492] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: No bootstrap required, opened a new log
I20240502 01:57:58.997020 21326 tablet_bootstrap.cc:492] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: No bootstrap required, opened a new log
I20240502 01:57:58.997094 21325 ts_tablet_manager.cc:1367] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:57:58.997176 21326 ts_tablet_manager.cc:1367] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:57:58.997176 21324 ts_tablet_manager.cc:1367] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 01:57:58.999763 21325 raft_consensus.cc:348] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:58.999943 21325 raft_consensus.cc:374] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:59.000149 21325 raft_consensus.cc:729] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a384193f264c0f975debcd152c98cf, State: Initialized, Role: FOLLOWER
I20240502 01:57:59.000602 21326 raft_consensus.cc:348] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.000602 21324 raft_consensus.cc:348] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.000658 21325 consensus_queue.cc:260] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.000797 21324 raft_consensus.cc:374] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:59.000797 21326 raft_consensus.cc:374] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:57:59.000852 21324 raft_consensus.cc:729] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4e33ce67ee42eab7d5997131ae5927, State: Initialized, Role: FOLLOWER
I20240502 01:57:59.000859 21326 raft_consensus.cc:729] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2eaeb7384cf4a95be01e4aef273d0dc, State: Initialized, Role: FOLLOWER
I20240502 01:57:59.001245 21326 consensus_queue.cc:260] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.001248 21324 consensus_queue.cc:260] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.001363 20902 heartbeater.cc:503] Master 127.19.43.254:41121 was elected leader, sending a full tablet report...
I20240502 01:57:59.001519 21325 ts_tablet_manager.cc:1398] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20240502 01:57:59.002009 21324 ts_tablet_manager.cc:1398] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 01:57:59.002009 21326 ts_tablet_manager.cc:1398] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 01:57:59.002375 21038 heartbeater.cc:503] Master 127.19.43.254:41121 was elected leader, sending a full tablet report...
I20240502 01:57:59.002554 20763 heartbeater.cc:503] Master 127.19.43.254:41121 was elected leader, sending a full tablet report...
W20240502 01:57:59.013803 20764 tablet.cc:2369] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:57:59.090982 21039 tablet.cc:2369] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:57:59.181517 20903 tablet.cc:2369] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:57:59.207890 21332 raft_consensus.cc:482] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:57:59.208132 21332 raft_consensus.cc:504] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.209476 21332 leader_election.cc:288] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2eaeb7384cf4a95be01e4aef273d0dc (127.19.43.195:36883), 72a384193f264c0f975debcd152c98cf (127.19.43.194:39033)
I20240502 01:57:59.213441 20990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77955270d5694232af40e471a331e796" candidate_uuid: "cd4e33ce67ee42eab7d5997131ae5927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" is_pre_election: true
I20240502 01:57:59.213634 20852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77955270d5694232af40e471a331e796" candidate_uuid: "cd4e33ce67ee42eab7d5997131ae5927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a384193f264c0f975debcd152c98cf" is_pre_election: true
I20240502 01:57:59.213773 20990 raft_consensus.cc:2454] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4e33ce67ee42eab7d5997131ae5927 in term 0.
I20240502 01:57:59.213887 20852 raft_consensus.cc:2454] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4e33ce67ee42eab7d5997131ae5927 in term 0.
I20240502 01:57:59.214238 20637 leader_election.cc:302] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4e33ce67ee42eab7d5997131ae5927, f2eaeb7384cf4a95be01e4aef273d0dc; no voters:
I20240502 01:57:59.214478 21332 raft_consensus.cc:2781] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:57:59.214555 21332 raft_consensus.cc:482] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:57:59.214599 21332 raft_consensus.cc:3037] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:59.215431 21332 raft_consensus.cc:504] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.215749 21332 leader_election.cc:288] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [CANDIDATE]: Term 1 election: Requested vote from peers f2eaeb7384cf4a95be01e4aef273d0dc (127.19.43.195:36883), 72a384193f264c0f975debcd152c98cf (127.19.43.194:39033)
I20240502 01:57:59.216091 20990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77955270d5694232af40e471a331e796" candidate_uuid: "cd4e33ce67ee42eab7d5997131ae5927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc"
I20240502 01:57:59.216143 20852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77955270d5694232af40e471a331e796" candidate_uuid: "cd4e33ce67ee42eab7d5997131ae5927" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a384193f264c0f975debcd152c98cf"
I20240502 01:57:59.216239 20852 raft_consensus.cc:3037] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:59.216234 20990 raft_consensus.cc:3037] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:57:59.217224 20990 raft_consensus.cc:2454] T 77955270d5694232af40e471a331e796 P f2eaeb7384cf4a95be01e4aef273d0dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4e33ce67ee42eab7d5997131ae5927 in term 1.
I20240502 01:57:59.217223 20852 raft_consensus.cc:2454] T 77955270d5694232af40e471a331e796 P 72a384193f264c0f975debcd152c98cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4e33ce67ee42eab7d5997131ae5927 in term 1.
I20240502 01:57:59.217512 20637 leader_election.cc:302] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4e33ce67ee42eab7d5997131ae5927, f2eaeb7384cf4a95be01e4aef273d0dc; no voters:
I20240502 01:57:59.217664 21332 raft_consensus.cc:2781] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:57:59.217875 21332 raft_consensus.cc:686] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [term 1 LEADER]: Becoming Leader. State: Replica: cd4e33ce67ee42eab7d5997131ae5927, State: Running, Role: LEADER
I20240502 01:57:59.218240 21332 consensus_queue.cc:237] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } }
I20240502 01:57:59.220938 20553 catalog_manager.cc:5484] T 77955270d5694232af40e471a331e796 P cd4e33ce67ee42eab7d5997131ae5927 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd4e33ce67ee42eab7d5997131ae5927 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "cd4e33ce67ee42eab7d5997131ae5927" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2eaeb7384cf4a95be01e4aef273d0dc" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 36883 } } peers { permanent_uuid: "72a384193f264c0f975debcd152c98cf" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 39033 } } peers { permanent_uuid: "cd4e33ce67ee42eab7d5997131ae5927" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 44263 } } }
I20240502 01:57:59.269327 19631 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 01:57:59.270717 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver cd4e33ce67ee42eab7d5997131ae5927 to finish bootstrapping
I20240502 01:57:59.275465 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 72a384193f264c0f975debcd152c98cf to finish bootstrapping
I20240502 01:57:59.279507 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver f2eaeb7384cf4a95be01e4aef273d0dc to finish bootstrapping
I20240502 01:57:59.283511 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver d389ebfa0fb841c3bca626080ea463f3 to finish bootstrapping
I20240502 01:57:59.288507 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 01c40b3831064a1aa20aae271625f4cc to finish bootstrapping
I20240502 01:57:59.293218 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20620
I20240502 01:57:59.478834 21238 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.479983 20831 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.480388 20967 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.482283 21104 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.554625 20831 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.554841 21238 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.555289 21104 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.555665 20967 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.645376 20831 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.646209 20967 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.646235 21104 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.647070 21238 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.775404 21173 heartbeater.cc:503] Master 127.19.43.254:41121 was elected leader, sending a full tablet report...
I20240502 01:57:59.846174 21238 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.847294 20831 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.847328 20967 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.847759 21104 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.915072 21238 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.915338 21104 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.916101 20831 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.916484 20967 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:57:59.955629 21307 heartbeater.cc:503] Master 127.19.43.254:41121 was elected leader, sending a full tablet report...
I20240502 01:58:00.010718 20831 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:00.011384 21238 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:00.012578 21104 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:00.013353 20967 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:00.045902 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20768
I20240502 01:58:00.051304 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20905
I20240502 01:58:00.056399 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 21041
I20240502 01:58:00.061062 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 21176
I20240502 01:58:00.065750 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 20524
2024-05-02T01:58:00Z chronyd exiting
[ OK ] RebalanceStartCriteriaTest.TabletServerIsDown/0 (2218 ms)
[----------] 1 test from RebalanceStartCriteriaTest (2218 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/0
I20240502 01:58:00.086467 19631 test_util.cc:268] Using random seed: -288986179
I20240502 01:58:00.087170 19631 ts_itest-base.cc:112] Starting cluster with:
I20240502 01:58:00.087221 19631 ts_itest-base.cc:113] --------------
I20240502 01:58:00.087231 19631 ts_itest-base.cc:114] 7 tablet servers
I20240502 01:58:00.087240 19631 ts_itest-base.cc:115] 1 replicas per TS
I20240502 01:58:00.087249 19631 ts_itest-base.cc:116] --------------
2024-05-02T01:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T01:58:00Z Disabled control of system clock
I20240502 01:58:00.094429 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:42289
--webserver_interface=127.19.43.254
--webserver_port=0
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.43.254:42289
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240502 01:58:00.196362 21530 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 01:58:00.196595 21530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:00.196620 21530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:00.196663 21530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:00.199637 21530 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:00.199703 21530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 01:58:00.199724 21530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:00.199749 21530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 01:58:00.199765 21530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 01:58:00.203339 21530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.19.43.254:42289
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:42289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.19.43.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:00.204185 21530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:00.205250 21530 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:00.211824 21535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.211833 21536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.212033 21530 server_base.cc:1034] running on GCE node
W20240502 01:58:00.211822 21538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.212512 21530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:00.213342 21530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:00.214478 21530 hybrid_clock.cc:648] HybridClock initialized: now 1714615080214459 us; error 33 us; skew 500 ppm
I20240502 01:58:00.216161 21530 webserver.cc:460] Webserver started at http://127.19.43.254:39601/ using document root <none> and password file <none>
I20240502 01:58:00.216653 21530 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:00.216706 21530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:00.216886 21530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:00.218466 21530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c1be09c507b24ad78d60f64e8e64d7a6"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.218919 21530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c1be09c507b24ad78d60f64e8e64d7a6"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.221876 21530 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:00.223831 21544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.224721 21530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:00.224830 21530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c1be09c507b24ad78d60f64e8e64d7a6"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.224905 21530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:00.234899 21530 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:00.235442 21530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:00.235577 21530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:00.241642 21530 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.254:42289
I20240502 01:58:00.241649 21598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.254:42289 every 8 connection(s)
I20240502 01:58:00.242558 21530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 01:58:00.244853 21599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:00.248268 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 21530
I20240502 01:58:00.248379 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 01:58:00.249723 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6: Bootstrap starting.
I20240502 01:58:00.251732 21599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:00.252446 21599 log.cc:824] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:00.253875 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6: No bootstrap required, opened a new log
I20240502 01:58:00.256240 21599 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 42289 } }
I20240502 01:58:00.256405 21599 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:00.256439 21599 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1be09c507b24ad78d60f64e8e64d7a6, State: Initialized, Role: FOLLOWER
I20240502 01:58:00.256847 21599 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 42289 } }
I20240502 01:58:00.256979 21599 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:00.257038 21599 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:00.257131 21599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:00.257778 21599 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 42289 } }
I20240502 01:58:00.258100 21599 leader_election.cc:302] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1be09c507b24ad78d60f64e8e64d7a6; no voters:
I20240502 01:58:00.258342 21599 leader_election.cc:288] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:00.258687 21604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:00.258870 21604 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [term 1 LEADER]: Becoming Leader. State: Replica: c1be09c507b24ad78d60f64e8e64d7a6, State: Running, Role: LEADER
I20240502 01:58:00.259122 21604 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 42289 } }
I20240502 01:58:00.259289 21599 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 01:58:00.260409 21607 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1be09c507b24ad78d60f64e8e64d7a6. Latest consensus state: current_term: 1 leader_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 42289 } } }
I20240502 01:58:00.260521 21607 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [sys.catalog]: This master's current role is: LEADER
I20240502 01:58:00.260780 21606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1be09c507b24ad78d60f64e8e64d7a6" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 42289 } } }
I20240502 01:58:00.260869 21606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6 [sys.catalog]: This master's current role is: LEADER
I20240502 01:58:00.260849 21614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 01:58:00.263403 21614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 01:58:00.267578 21614 catalog_manager.cc:1347] Generated new cluster ID: 9ed2a978b0764f8d8b3bf8d4c6f86bfc
I20240502 01:58:00.267639 21614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 01:58:00.294376 21614 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 01:58:00.295360 21614 catalog_manager.cc:1504] Loading token signing keys...
I20240502 01:58:00.309039 21614 catalog_manager.cc:5857] T 00000000000000000000000000000000 P c1be09c507b24ad78d60f64e8e64d7a6: Generated new TSK 0
I20240502 01:58:00.309792 21614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 01:58:00.313213 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.193:0
--local_ip_for_outbound_sockets=127.19.43.193
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:00.428112 21624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:00.428293 21624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:00.428346 21624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:00.431350 21624 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:00.431409 21624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:00.431479 21624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.193
I20240502 01:58:00.434950 21624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.21624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.193
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:00.435735 21624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:00.436770 21624 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:00.443842 21629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.443859 21630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.443853 21632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.444021 21624 server_base.cc:1034] running on GCE node
I20240502 01:58:00.444454 21624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:00.444959 21624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:00.446084 21624 hybrid_clock.cc:648] HybridClock initialized: now 1714615080446064 us; error 32 us; skew 500 ppm
I20240502 01:58:00.447679 21624 webserver.cc:460] Webserver started at http://127.19.43.193:46751/ using document root <none> and password file <none>
I20240502 01:58:00.448211 21624 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:00.448261 21624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:00.448429 21624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:00.449978 21624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1ab91a4d0007452a85f25102557ed3dd"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.450448 21624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1ab91a4d0007452a85f25102557ed3dd"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.453420 21624 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:00.455402 21639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.456319 21624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:00.456426 21624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1ab91a4d0007452a85f25102557ed3dd"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.456501 21624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:00.472298 21624 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:00.472880 21624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:00.473022 21624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:00.473512 21624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:00.474391 21624 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:00.474437 21624 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.474488 21624 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:00.474516 21624 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.482338 21624 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.193:42561
I20240502 01:58:00.482364 21757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.193:42561 every 8 connection(s)
I20240502 01:58:00.483178 21624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 01:58:00.488646 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 21624
I20240502 01:58:00.488744 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 01:58:00.490814 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.194:0
--local_ip_for_outbound_sockets=127.19.43.194
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:00.491410 21758 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:00.491629 21758 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:00.492116 21758 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:00.493503 21563 ts_manager.cc:194] Registered new tserver with Master: 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:00.495070 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.193:60697
W20240502 01:58:00.596740 21761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:00.596930 21761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:00.596971 21761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:00.599889 21761 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:00.599943 21761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:00.600010 21761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.194
I20240502 01:58:00.603533 21761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.21761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.194
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:00.604326 21761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:00.605371 21761 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:00.612706 21766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.612777 21769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.612708 21767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.612838 21761 server_base.cc:1034] running on GCE node
I20240502 01:58:00.613294 21761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:00.613822 21761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:00.614972 21761 hybrid_clock.cc:648] HybridClock initialized: now 1714615080614950 us; error 36 us; skew 500 ppm
I20240502 01:58:00.616658 21761 webserver.cc:460] Webserver started at http://127.19.43.194:43033/ using document root <none> and password file <none>
I20240502 01:58:00.617158 21761 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:00.617210 21761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:00.617386 21761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:00.618999 21761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c5e97283120f47719b7fcec14be91dfe"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.619452 21761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c5e97283120f47719b7fcec14be91dfe"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.622524 21761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20240502 01:58:00.624526 21776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.625465 21761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:00.625553 21761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c5e97283120f47719b7fcec14be91dfe"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.625633 21761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:00.639611 21761 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:00.640174 21761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:00.640313 21761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:00.640797 21761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:00.641701 21761 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:00.641760 21761 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.641816 21761 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:00.641844 21761 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.649713 21761 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.194:43977
I20240502 01:58:00.649760 21891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.194:43977 every 8 connection(s)
I20240502 01:58:00.650614 21761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 01:58:00.655901 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 21761
I20240502 01:58:00.656003 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 01:58:00.658115 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.195:0
--local_ip_for_outbound_sockets=127.19.43.195
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:00.658829 21892 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:00.659022 21892 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:00.659519 21892 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:00.660519 21563 ts_manager.cc:194] Registered new tserver with Master: c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:00.661110 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.194:57043
W20240502 01:58:00.760241 21895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:00.760478 21895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:00.760538 21895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:00.763593 21895 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:00.763654 21895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:00.763734 21895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.195
I20240502 01:58:00.767246 21895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.21895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.195
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:00.768033 21895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:00.769079 21895 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:00.776298 21903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.776248 21900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.776265 21901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.776510 21895 server_base.cc:1034] running on GCE node
I20240502 01:58:00.776960 21895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:00.777464 21895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:00.778600 21895 hybrid_clock.cc:648] HybridClock initialized: now 1714615080778587 us; error 25 us; skew 500 ppm
I20240502 01:58:00.780186 21895 webserver.cc:460] Webserver started at http://127.19.43.195:39085/ using document root <none> and password file <none>
I20240502 01:58:00.780700 21895 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:00.780753 21895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:00.780929 21895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:00.782541 21895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f32583d958864898bc7e3567aed4334e"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.782968 21895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f32583d958864898bc7e3567aed4334e"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.785961 21895 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:00.788053 21909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.788975 21895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:00.789073 21895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f32583d958864898bc7e3567aed4334e"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.789146 21895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:00.803889 21895 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:00.804499 21895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:00.804647 21895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:00.805151 21895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:00.806011 21895 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:00.806084 21895 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.806138 21895 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:00.806161 21895 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.814462 21895 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.195:42143
I20240502 01:58:00.814497 22026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.195:42143 every 8 connection(s)
I20240502 01:58:00.815325 21895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 01:58:00.823035 22027 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:00.823228 22027 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:00.823634 22027 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:00.823792 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 21895
I20240502 01:58:00.823886 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 01:58:00.824561 21563 ts_manager.cc:194] Registered new tserver with Master: f32583d958864898bc7e3567aed4334e (127.19.43.195:42143)
I20240502 01:58:00.825088 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.195:54665
I20240502 01:58:00.825879 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.196:0
--local_ip_for_outbound_sockets=127.19.43.196
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:00.927637 22030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:00.927839 22030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:00.927881 22030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:00.930927 22030 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:00.930985 22030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:00.931048 22030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.196
I20240502 01:58:00.934545 22030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.22030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.196
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:00.935344 22030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:00.936407 22030 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:00.943634 22035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:00.943616 22036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.943743 22030 server_base.cc:1034] running on GCE node
W20240502 01:58:00.943610 22038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:00.944169 22030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:00.944679 22030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:00.945825 22030 hybrid_clock.cc:648] HybridClock initialized: now 1714615080945810 us; error 33 us; skew 500 ppm
I20240502 01:58:00.947554 22030 webserver.cc:460] Webserver started at http://127.19.43.196:33467/ using document root <none> and password file <none>
I20240502 01:58:00.948051 22030 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:00.948107 22030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:00.948266 22030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:00.949805 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "90c9a86c10804941bfe2590985250c75"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.950265 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "90c9a86c10804941bfe2590985250c75"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.953277 22030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:00.955302 22045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.956218 22030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:00.956318 22030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "90c9a86c10804941bfe2590985250c75"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:00.956399 22030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:00.968973 22030 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:00.969588 22030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:00.969751 22030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:00.970296 22030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:00.971169 22030 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:00.971226 22030 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.971277 22030 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:00.971304 22030 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:00.979414 22162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.196:42373 every 8 connection(s)
I20240502 01:58:00.979398 22030 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.196:42373
I20240502 01:58:00.980331 22030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 01:58:00.988749 22163 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:00.988966 22163 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:00.989377 22163 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:00.990262 21563 ts_manager.cc:194] Registered new tserver with Master: 90c9a86c10804941bfe2590985250c75 (127.19.43.196:42373)
I20240502 01:58:00.990535 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 22030
I20240502 01:58:00.990630 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 01:58:00.990857 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.196:60405
I20240502 01:58:00.992600 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:0
--local_ip_for_outbound_sockets=127.19.43.197
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:01.095031 22167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:01.095235 22167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:01.095278 22167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:01.098276 22167 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:01.098340 22167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:01.098430 22167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:58:01.101905 22167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.22167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:01.102763 22167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:01.103844 22167 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:01.111255 22175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:01.111239 22173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:01.111239 22172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:01.111704 22167 server_base.cc:1034] running on GCE node
I20240502 01:58:01.112035 22167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:01.112522 22167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:01.113679 22167 hybrid_clock.cc:648] HybridClock initialized: now 1714615081113658 us; error 33 us; skew 500 ppm
I20240502 01:58:01.115491 22167 webserver.cc:460] Webserver started at http://127.19.43.197:35247/ using document root <none> and password file <none>
I20240502 01:58:01.116022 22167 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:01.116078 22167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:01.116254 22167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:01.117861 22167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "157390243b4541548352b9d27da66cb7"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.118336 22167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "157390243b4541548352b9d27da66cb7"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.122166 22167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20240502 01:58:01.124267 22181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.125195 22167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:01.125300 22167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "157390243b4541548352b9d27da66cb7"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.125378 22167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:01.141927 22167 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:01.142534 22167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:01.142692 22167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:01.143201 22167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:01.144070 22167 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:01.144119 22167 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.144170 22167 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:01.144204 22167 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.152139 22167 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:45485
I20240502 01:58:01.152161 22297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:45485 every 8 connection(s)
I20240502 01:58:01.153019 22167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:58:01.157795 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 22167
I20240502 01:58:01.157902 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 01:58:01.160137 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.198:0
--local_ip_for_outbound_sockets=127.19.43.198
--webserver_interface=127.19.43.198
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:01.161921 22298 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:01.162174 22298 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:01.162597 22298 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:01.163537 21563 ts_manager.cc:194] Registered new tserver with Master: 157390243b4541548352b9d27da66cb7 (127.19.43.197:45485)
I20240502 01:58:01.164253 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:33609
W20240502 01:58:01.263736 22301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:01.263931 22301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:01.263973 22301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:01.266927 22301 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:01.266984 22301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:01.267051 22301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.198
I20240502 01:58:01.270494 22301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.19.43.198
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.22301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.198
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:01.271274 22301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:01.272301 22301 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:01.279583 22306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:01.279632 22309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:01.279640 22307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:01.279855 22301 server_base.cc:1034] running on GCE node
I20240502 01:58:01.280205 22301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:01.280718 22301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:01.281864 22301 hybrid_clock.cc:648] HybridClock initialized: now 1714615081281850 us; error 25 us; skew 500 ppm
I20240502 01:58:01.283663 22301 webserver.cc:460] Webserver started at http://127.19.43.198:38421/ using document root <none> and password file <none>
I20240502 01:58:01.284178 22301 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:01.284245 22301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:01.284410 22301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:01.286094 22301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "d74b9a09ea17435298a8a6f2b82281e0"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.286563 22301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "d74b9a09ea17435298a8a6f2b82281e0"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.289783 22301 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:01.291879 22316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.292886 22301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:01.292994 22301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "d74b9a09ea17435298a8a6f2b82281e0"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.293071 22301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:01.306805 22301 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:01.307430 22301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:01.307577 22301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:01.308075 22301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:01.308930 22301 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:01.308975 22301 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.309027 22301 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:01.309053 22301 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.317400 22301 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.198:35551
I20240502 01:58:01.317417 22436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.198:35551 every 8 connection(s)
I20240502 01:58:01.318295 22301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 01:58:01.325178 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 22301
I20240502 01:58:01.325282 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 01:58:01.326521 22437 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:01.326728 22437 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:01.327227 22437 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:01.327486 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.199:0
--local_ip_for_outbound_sockets=127.19.43.199
--webserver_interface=127.19.43.199
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:01.328212 21563 ts_manager.cc:194] Registered new tserver with Master: d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198:35551)
I20240502 01:58:01.328971 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.198:41683
W20240502 01:58:01.431339 22441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:01.431541 22441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:01.431587 22441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:01.434599 22441 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:01.434662 22441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:01.434756 22441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.199
I20240502 01:58:01.438277 22441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.19.43.199
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.22441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.199
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:01.439114 22441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:01.440200 22441 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:01.447662 22447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:01.447666 22449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:01.447666 22446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:01.447955 22441 server_base.cc:1034] running on GCE node
I20240502 01:58:01.448288 22441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:01.448793 22441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:01.449940 22441 hybrid_clock.cc:648] HybridClock initialized: now 1714615081449922 us; error 31 us; skew 500 ppm
I20240502 01:58:01.451576 22441 webserver.cc:460] Webserver started at http://127.19.43.199:44207/ using document root <none> and password file <none>
I20240502 01:58:01.452081 22441 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:01.452139 22441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:01.452299 22441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:01.453871 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "9e6dd740161e46459fdb27c45ae2498e"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.454353 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "9e6dd740161e46459fdb27c45ae2498e"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.457444 22441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:01.459601 22455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.460634 22441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:01.460729 22441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "9e6dd740161e46459fdb27c45ae2498e"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:01.460809 22441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:01.480768 22441 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:01.481393 22441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:01.481540 22441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:01.482075 22441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:01.482997 22441 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:01.483044 22441 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.483095 22441 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:01.483119 22441 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:01.491503 22441 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.199:44415
I20240502 01:58:01.491531 22574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.199:44415 every 8 connection(s)
I20240502 01:58:01.492379 22441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 01:58:01.493103 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 22441
I20240502 01:58:01.493202 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 01:58:01.497160 21758 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:01.500726 22575 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:01.500900 22575 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:01.501264 22575 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:01.502067 21563 ts_manager.cc:194] Registered new tserver with Master: 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199:44415)
I20240502 01:58:01.502673 21563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.199:42703
I20240502 01:58:01.507014 19631 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 01:58:01.515228 21563 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49428:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 01:58:01.530540 21825 tablet_service.cc:1467] Processing CreateTablet for tablet 97be852ca8c241c1a44c3e4a66490ca9 (DEFAULT_TABLE table=TestTable [id=28378353a9a94c14bb536524c855d738]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:01.531378 21825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97be852ca8c241c1a44c3e4a66490ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:01.536373 22589 tablet_bootstrap.cc:492] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:01.538168 22589 tablet_bootstrap.cc:654] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:01.538806 22589 log.cc:824] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:01.540236 22589 tablet_bootstrap.cc:492] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:01.540409 22589 ts_tablet_manager.cc:1367] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240502 01:58:01.542716 22589 raft_consensus.cc:348] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:01.542881 22589 raft_consensus.cc:374] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:01.542928 22589 raft_consensus.cc:729] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:01.543293 22589 consensus_queue.cc:260] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:01.543416 22589 raft_consensus.cc:388] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:01.543474 22589 raft_consensus.cc:482] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:01.543572 22589 raft_consensus.cc:3037] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:01.544589 22589 raft_consensus.cc:504] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:01.544880 22589 leader_election.cc:302] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:01.545130 22589 leader_election.cc:288] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:01.545222 22591 raft_consensus.cc:2781] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:01.545426 22591 raft_consensus.cc:686] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:01.545715 21892 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:01.545706 22591 consensus_queue.cc:237] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:01.545918 22589 ts_tablet_manager.cc:1398] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20240502 01:58:01.548564 21563 catalog_manager.cc:5484] T 97be852ca8c241c1a44c3e4a66490ca9 P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
I20240502 01:58:01.555773 19631 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 01:58:01.556946 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 1ab91a4d0007452a85f25102557ed3dd to finish bootstrapping
I20240502 01:58:01.562534 19631 ts_itest-base.cc:233] Waiting for 1 tablets on tserver c5e97283120f47719b7fcec14be91dfe to finish bootstrapping
I20240502 01:58:01.566519 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver f32583d958864898bc7e3567aed4334e to finish bootstrapping
I20240502 01:58:01.571435 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 90c9a86c10804941bfe2590985250c75 to finish bootstrapping
I20240502 01:58:01.576371 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 157390243b4541548352b9d27da66cb7 to finish bootstrapping
I20240502 01:58:01.581269 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver d74b9a09ea17435298a8a6f2b82281e0 to finish bootstrapping
I20240502 01:58:01.586115 19631 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 9e6dd740161e46459fdb27c45ae2498e to finish bootstrapping
I20240502 01:58:01.591168 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 21895
I20240502 01:58:01.596917 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 22030
I20240502 01:58:01.601675 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 22167
I20240502 01:58:01.607010 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 22301
I20240502 01:58:01.611958 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 22441
I20240502 01:58:05.368844 21563 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49428:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:05.380319 21689 tablet_service.cc:1467] Processing CreateTablet for tablet f0b94526b4414b758214e8d3800d2b53 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6fcab502c66b4b9cabfc4baaaa7dad37]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:05.380328 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 39b14aa4387a4d30854f4886383a9452 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6fcab502c66b4b9cabfc4baaaa7dad37]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:05.380331 21687 tablet_service.cc:1467] Processing CreateTablet for tablet b420b13c524544cd8560a93a7d5b6ee1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6fcab502c66b4b9cabfc4baaaa7dad37]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:05.381520 21689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b94526b4414b758214e8d3800d2b53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:05.381839 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420b13c524544cd8560a93a7d5b6ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:05.382061 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b14aa4387a4d30854f4886383a9452. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:05.386837 22610 tablet_bootstrap.cc:492] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:05.388957 22610 tablet_bootstrap.cc:654] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:05.389577 22610 log.cc:824] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:05.391000 22610 tablet_bootstrap.cc:492] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:05.391140 22610 ts_tablet_manager.cc:1367] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:58:05.393548 22610 raft_consensus.cc:348] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.393716 22610 raft_consensus.cc:374] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:05.393756 22610 raft_consensus.cc:729] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:05.394152 22610 consensus_queue.cc:260] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.394265 22610 raft_consensus.cc:388] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:05.394309 22610 raft_consensus.cc:482] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:05.394385 22610 raft_consensus.cc:3037] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:05.454008 22610 raft_consensus.cc:504] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.454488 22610 leader_election.cc:302] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:05.454826 22610 leader_election.cc:288] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:05.454970 22613 raft_consensus.cc:2781] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:05.455183 22613 raft_consensus.cc:686] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:05.455741 22610 ts_tablet_manager.cc:1398] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.065s user 0.007s sys 0.000s
I20240502 01:58:05.455852 22613 consensus_queue.cc:237] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.455999 22610 tablet_bootstrap.cc:492] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:05.456858 22610 tablet_bootstrap.cc:654] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:05.457948 22610 tablet_bootstrap.cc:492] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:05.458079 22610 ts_tablet_manager.cc:1367] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:05.458627 22610 raft_consensus.cc:348] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.458729 22610 raft_consensus.cc:374] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:05.458765 22610 raft_consensus.cc:729] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:05.458891 22610 consensus_queue.cc:260] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.458967 22610 raft_consensus.cc:388] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:05.458989 22610 raft_consensus.cc:482] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:05.459031 22610 raft_consensus.cc:3037] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:05.459139 21563 catalog_manager.cc:5484] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
I20240502 01:58:05.557947 22610 raft_consensus.cc:504] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.558208 22610 leader_election.cc:302] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:05.558319 22610 leader_election.cc:288] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:05.558393 22613 raft_consensus.cc:2781] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:05.558571 22610 ts_tablet_manager.cc:1398] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.100s user 0.002s sys 0.000s
I20240502 01:58:05.558745 22610 tablet_bootstrap.cc:492] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:05.558791 22613 raft_consensus.cc:686] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:05.558925 22613 consensus_queue.cc:237] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.560246 22610 tablet_bootstrap.cc:654] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:05.560567 21563 catalog_manager.cc:5484] T 39b14aa4387a4d30854f4886383a9452 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
I20240502 01:58:05.561646 22610 tablet_bootstrap.cc:492] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:05.561743 22610 ts_tablet_manager.cc:1367] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:05.562204 22610 raft_consensus.cc:348] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.562304 22610 raft_consensus.cc:374] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:05.562335 22610 raft_consensus.cc:729] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:05.562449 22610 consensus_queue.cc:260] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.562542 22610 raft_consensus.cc:388] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:05.562582 22610 raft_consensus.cc:482] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:05.562628 22610 raft_consensus.cc:3037] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:05.659163 22610 raft_consensus.cc:504] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.659363 22610 leader_election.cc:302] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:05.659471 22610 leader_election.cc:288] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:05.659595 22613 raft_consensus.cc:2781] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:05.659758 22613 raft_consensus.cc:686] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:05.659821 22610 ts_tablet_manager.cc:1398] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.098s user 0.002s sys 0.000s
I20240502 01:58:05.659891 22613 consensus_queue.cc:237] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:05.661540 21563 catalog_manager.cc:5484] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
W20240502 01:58:05.773054 22623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:05.773319 22623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:05.776348 22623 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
INSERT report
rows total: 2000
time total: 60.6006 ms
time per row: 0.0303003 ms
I20240502 01:58:05.881567 21562 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49428:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:05.887063 21824 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac53985ac124e23ac10c86b2b048aa8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=39421104a2574a2ab88931f1d0b154e9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:05.887063 21823 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef85fe1e5c4dc1bb810d3c93742756 (DEFAULT_TABLE table=rebalance_test_table_1 [id=39421104a2574a2ab88931f1d0b154e9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:05.887063 21825 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3ca0726e154d74aa3da3afc065c827 (DEFAULT_TABLE table=rebalance_test_table_1 [id=39421104a2574a2ab88931f1d0b154e9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:05.887455 21824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac53985ac124e23ac10c86b2b048aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:05.887626 21825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a3ca0726e154d74aa3da3afc065c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:05.887763 21823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef85fe1e5c4dc1bb810d3c93742756. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:05.889640 22655 tablet_bootstrap.cc:492] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:05.890622 22655 tablet_bootstrap.cc:654] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:05.891551 22655 tablet_bootstrap.cc:492] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:05.891620 22655 ts_tablet_manager.cc:1367] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:05.892069 22655 raft_consensus.cc:348] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:05.892160 22655 raft_consensus.cc:374] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:05.892189 22655 raft_consensus.cc:729] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:05.892313 22655 consensus_queue.cc:260] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:05.892390 22655 raft_consensus.cc:388] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:05.892411 22655 raft_consensus.cc:482] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:05.892454 22655 raft_consensus.cc:3037] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:05.965011 22655 raft_consensus.cc:504] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:05.965304 22655 leader_election.cc:302] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:05.965525 22655 leader_election.cc:288] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:05.965683 22657 raft_consensus.cc:2781] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:05.965843 22655 ts_tablet_manager.cc:1398] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.074s user 0.003s sys 0.000s
I20240502 01:58:05.966045 22655 tablet_bootstrap.cc:492] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:05.966140 22657 raft_consensus.cc:686] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:05.966315 22657 consensus_queue.cc:237] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:05.967070 22655 tablet_bootstrap.cc:654] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:05.967991 21562 catalog_manager.cc:5484] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
I20240502 01:58:05.968938 22655 tablet_bootstrap.cc:492] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:05.969031 22655 ts_tablet_manager.cc:1367] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:05.969475 22655 raft_consensus.cc:348] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:05.969563 22655 raft_consensus.cc:374] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:05.969592 22655 raft_consensus.cc:729] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:05.969765 22655 consensus_queue.cc:260] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:05.969873 22655 raft_consensus.cc:388] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:05.969918 22655 raft_consensus.cc:482] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:05.969969 22655 raft_consensus.cc:3037] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:06.065927 22655 raft_consensus.cc:504] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.066183 22655 leader_election.cc:302] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:06.066309 22655 leader_election.cc:288] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:06.066396 22657 raft_consensus.cc:2781] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:06.066519 22657 raft_consensus.cc:686] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:06.066655 22657 consensus_queue.cc:237] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.066692 22655 ts_tablet_manager.cc:1398] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.098s user 0.003s sys 0.000s
I20240502 01:58:06.066893 22655 tablet_bootstrap.cc:492] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:06.068327 22655 tablet_bootstrap.cc:654] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:06.068514 21562 catalog_manager.cc:5484] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
I20240502 01:58:06.069322 22655 tablet_bootstrap.cc:492] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:06.069411 22655 ts_tablet_manager.cc:1367] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:06.069828 22655 raft_consensus.cc:348] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.069909 22655 raft_consensus.cc:374] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:06.069938 22655 raft_consensus.cc:729] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:06.070080 22655 consensus_queue.cc:260] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.070153 22655 raft_consensus.cc:388] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:06.070183 22655 raft_consensus.cc:482] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:06.070232 22655 raft_consensus.cc:3037] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:06.169374 22655 raft_consensus.cc:504] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.169595 22655 leader_election.cc:302] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:06.169706 22655 leader_election.cc:288] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:06.169766 22657 raft_consensus.cc:2781] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:06.169884 22657 raft_consensus.cc:686] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:06.169970 22655 ts_tablet_manager.cc:1398] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.101s user 0.003s sys 0.000s
I20240502 01:58:06.170044 22657 consensus_queue.cc:237] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.171525 21562 catalog_manager.cc:5484] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
W20240502 01:58:06.279493 22666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:06.279739 22666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:06.282790 22666 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
INSERT report
rows total: 2000
time total: 59.0371 ms
time per row: 0.0295185 ms
I20240502 01:58:06.381669 21556 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49428:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:06.386987 21689 tablet_service.cc:1467] Processing CreateTablet for tablet c03ce899e48c4447b56b1353c3f1d818 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e32c50be8d624374b017ee7e8ced8664]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:06.387025 21688 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2a1efb8da4f32ad8a2e74610892dd (DEFAULT_TABLE table=rebalance_test_table_2 [id=e32c50be8d624374b017ee7e8ced8664]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:06.386987 21687 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd57c7e02a74d5988cf68d7546ca770 (DEFAULT_TABLE table=rebalance_test_table_2 [id=e32c50be8d624374b017ee7e8ced8664]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:06.387372 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2a1efb8da4f32ad8a2e74610892dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:06.387501 21689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03ce899e48c4447b56b1353c3f1d818. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:06.388550 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd57c7e02a74d5988cf68d7546ca770. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:06.389593 22699 tablet_bootstrap.cc:492] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:06.390450 22699 tablet_bootstrap.cc:654] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:06.391265 22699 tablet_bootstrap.cc:492] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:06.391345 22699 ts_tablet_manager.cc:1367] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:06.391705 22699 raft_consensus.cc:348] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.391794 22699 raft_consensus.cc:374] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:06.391815 22699 raft_consensus.cc:729] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:06.391906 22699 consensus_queue.cc:260] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.391960 22699 raft_consensus.cc:388] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:06.391989 22699 raft_consensus.cc:482] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:06.392021 22699 raft_consensus.cc:3037] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:06.429013 22699 raft_consensus.cc:504] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.429224 22699 leader_election.cc:302] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:06.429437 22699 leader_election.cc:288] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:06.429623 22701 raft_consensus.cc:2781] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:06.429838 22701 raft_consensus.cc:686] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:06.429986 22701 consensus_queue.cc:237] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.430150 22699 ts_tablet_manager.cc:1398] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20240502 01:58:06.430332 22699 tablet_bootstrap.cc:492] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:06.431190 22699 tablet_bootstrap.cc:654] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:06.431536 21556 catalog_manager.cc:5484] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
I20240502 01:58:06.432166 22699 tablet_bootstrap.cc:492] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:06.432245 22699 ts_tablet_manager.cc:1367] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:06.432602 22699 raft_consensus.cc:348] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.432699 22699 raft_consensus.cc:374] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:06.432736 22699 raft_consensus.cc:729] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:06.432858 22699 consensus_queue.cc:260] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.432951 22699 raft_consensus.cc:388] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:06.432989 22699 raft_consensus.cc:482] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:06.433033 22699 raft_consensus.cc:3037] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:06.528733 22699 raft_consensus.cc:504] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.528993 22699 leader_election.cc:302] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:06.529122 22699 leader_election.cc:288] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:06.529234 22703 raft_consensus.cc:2781] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:06.529362 22703 raft_consensus.cc:686] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:06.529412 22699 ts_tablet_manager.cc:1398] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.097s user 0.003s sys 0.000s
I20240502 01:58:06.529508 22703 consensus_queue.cc:237] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.529654 22699 tablet_bootstrap.cc:492] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:06.530640 22699 tablet_bootstrap.cc:654] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:06.531373 21556 catalog_manager.cc:5484] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
I20240502 01:58:06.531592 22699 tablet_bootstrap.cc:492] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:06.531682 22699 ts_tablet_manager.cc:1367] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:06.532063 22699 raft_consensus.cc:348] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.532163 22699 raft_consensus.cc:374] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:06.532193 22699 raft_consensus.cc:729] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:06.532311 22699 consensus_queue.cc:260] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.532395 22699 raft_consensus.cc:388] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:06.532430 22699 raft_consensus.cc:482] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:06.532474 22699 raft_consensus.cc:3037] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:06.630393 22699 raft_consensus.cc:504] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.630625 22699 leader_election.cc:302] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:06.630739 22699 leader_election.cc:288] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:06.630807 22702 raft_consensus.cc:2781] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:06.630904 22702 raft_consensus.cc:686] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:06.631069 22702 consensus_queue.cc:237] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:06.631319 22699 ts_tablet_manager.cc:1398] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.100s user 0.003s sys 0.000s
I20240502 01:58:06.632871 21556 catalog_manager.cc:5484] T 3bd57c7e02a74d5988cf68d7546ca770 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
W20240502 01:58:06.787847 22711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:06.788117 22711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:06.791275 22711 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
INSERT report
rows total: 2000
time total: 67.6536 ms
time per row: 0.0338268 ms
I20240502 01:58:06.899561 21555 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49428:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:06.904758 21825 tablet_service.cc:1467] Processing CreateTablet for tablet 5d99be4a87b24089bc4185904dc8fd30 (DEFAULT_TABLE table=rebalance_test_table_3 [id=a7aaed55cbdd49a89a6c397d097bb628]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:06.904793 21823 tablet_service.cc:1467] Processing CreateTablet for tablet 8db3067333544aaf92267389358a6fbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=a7aaed55cbdd49a89a6c397d097bb628]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:06.904778 21824 tablet_service.cc:1467] Processing CreateTablet for tablet f37d05f42cca45018b116b1729755c52 (DEFAULT_TABLE table=rebalance_test_table_3 [id=a7aaed55cbdd49a89a6c397d097bb628]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:06.905121 21825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d99be4a87b24089bc4185904dc8fd30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:06.905280 21823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db3067333544aaf92267389358a6fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:06.905429 21824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37d05f42cca45018b116b1729755c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:06.907137 22744 tablet_bootstrap.cc:492] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:06.907980 22744 tablet_bootstrap.cc:654] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:06.908959 22744 tablet_bootstrap.cc:492] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:06.909029 22744 ts_tablet_manager.cc:1367] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:06.909364 22744 raft_consensus.cc:348] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.909449 22744 raft_consensus.cc:374] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:06.909471 22744 raft_consensus.cc:729] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:06.909590 22744 consensus_queue.cc:260] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.909689 22744 raft_consensus.cc:388] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:06.909736 22744 raft_consensus.cc:482] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:06.909782 22744 raft_consensus.cc:3037] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:06.937992 22744 raft_consensus.cc:504] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.938238 22744 leader_election.cc:302] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:06.938427 22744 leader_election.cc:288] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:06.938570 22747 raft_consensus.cc:2781] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:06.938786 22744 ts_tablet_manager.cc:1398] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.030s user 0.001s sys 0.002s
I20240502 01:58:06.939008 22744 tablet_bootstrap.cc:492] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:06.938784 22747 raft_consensus.cc:686] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:06.939321 22747 consensus_queue.cc:237] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.939898 22744 tablet_bootstrap.cc:654] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:06.940816 22744 tablet_bootstrap.cc:492] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:06.940801 21555 catalog_manager.cc:5484] T 5d99be4a87b24089bc4185904dc8fd30 P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
I20240502 01:58:06.940907 22744 ts_tablet_manager.cc:1367] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:06.941268 22744 raft_consensus.cc:348] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.941346 22744 raft_consensus.cc:374] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:06.941375 22744 raft_consensus.cc:729] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:06.941484 22744 consensus_queue.cc:260] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:06.941558 22744 raft_consensus.cc:388] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:06.941587 22744 raft_consensus.cc:482] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:06.941632 22744 raft_consensus.cc:3037] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:07.039613 22744 raft_consensus.cc:504] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:07.039871 22744 leader_election.cc:302] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:07.039983 22744 leader_election.cc:288] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:07.040083 22747 raft_consensus.cc:2781] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:07.040196 22747 raft_consensus.cc:686] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:07.040231 22744 ts_tablet_manager.cc:1398] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.099s user 0.002s sys 0.000s
I20240502 01:58:07.040338 22747 consensus_queue.cc:237] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:07.040426 22744 tablet_bootstrap.cc:492] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Bootstrap starting.
I20240502 01:58:07.041357 22744 tablet_bootstrap.cc:654] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:07.041976 21560 catalog_manager.cc:5484] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
I20240502 01:58:07.042531 22744 tablet_bootstrap.cc:492] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: No bootstrap required, opened a new log
I20240502 01:58:07.042629 22744 ts_tablet_manager.cc:1367] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:07.043045 22744 raft_consensus.cc:348] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:07.043145 22744 raft_consensus.cc:374] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:07.043186 22744 raft_consensus.cc:729] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Initialized, Role: FOLLOWER
I20240502 01:58:07.043309 22744 consensus_queue.cc:260] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:07.043483 22744 raft_consensus.cc:388] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:07.043525 22744 raft_consensus.cc:482] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:07.043565 22744 raft_consensus.cc:3037] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:07.044348 22744 raft_consensus.cc:504] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:07.044469 22744 leader_election.cc:302] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:07.044553 22744 leader_election.cc:288] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:07.044617 22747 raft_consensus.cc:2781] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:07.044739 22747 raft_consensus.cc:686] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Leader. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:07.044754 22744 ts_tablet_manager.cc:1398] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:07.044891 22747 consensus_queue.cc:237] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:07.046212 21560 catalog_manager.cc:5484] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e97283120f47719b7fcec14be91dfe (127.19.43.194). New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } }
W20240502 01:58:07.160943 22756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:07.161192 22756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:07.164146 22756 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
INSERT report
rows total: 2000
time total: 72.2419 ms
time per row: 0.0361209 ms
I20240502 01:58:07.274910 21560 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49428:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:07.280253 21687 tablet_service.cc:1467] Processing CreateTablet for tablet 630104b6bc4c49428f5a49561e396751 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e370f340e5c14b8a8efd4f0eade2363f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:07.280299 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 8580c51fd90541c8bf70a2cee9c9ce36 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e370f340e5c14b8a8efd4f0eade2363f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:07.280299 21689 tablet_service.cc:1467] Processing CreateTablet for tablet 6935b7c50570412691bb3086b2b7a370 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e370f340e5c14b8a8efd4f0eade2363f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:07.280579 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630104b6bc4c49428f5a49561e396751. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:07.280701 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8580c51fd90541c8bf70a2cee9c9ce36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:07.280870 21689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6935b7c50570412691bb3086b2b7a370. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:07.282552 22787 tablet_bootstrap.cc:492] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:07.283432 22787 tablet_bootstrap.cc:654] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:07.284266 22787 tablet_bootstrap.cc:492] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:07.284353 22787 ts_tablet_manager.cc:1367] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:07.284695 22787 raft_consensus.cc:348] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.284782 22787 raft_consensus.cc:374] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:07.284811 22787 raft_consensus.cc:729] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:07.284929 22787 consensus_queue.cc:260] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.284993 22787 raft_consensus.cc:388] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:07.285030 22787 raft_consensus.cc:482] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:07.285077 22787 raft_consensus.cc:3037] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:07.345095 22787 raft_consensus.cc:504] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.345297 22787 leader_election.cc:302] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:07.345404 22787 leader_election.cc:288] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:07.345463 22703 raft_consensus.cc:2781] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:07.345546 22703 raft_consensus.cc:686] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:07.345679 22787 ts_tablet_manager.cc:1398] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.061s user 0.002s sys 0.000s
I20240502 01:58:07.345685 22703 consensus_queue.cc:237] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.345865 22787 tablet_bootstrap.cc:492] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:07.347115 22787 tablet_bootstrap.cc:654] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:07.347369 21560 catalog_manager.cc:5484] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
I20240502 01:58:07.348202 22787 tablet_bootstrap.cc:492] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:07.348296 22787 ts_tablet_manager.cc:1367] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:07.348692 22787 raft_consensus.cc:348] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.348795 22787 raft_consensus.cc:374] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:07.348845 22787 raft_consensus.cc:729] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:07.348964 22787 consensus_queue.cc:260] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.349035 22787 raft_consensus.cc:388] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:07.349071 22787 raft_consensus.cc:482] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:07.349120 22787 raft_consensus.cc:3037] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:07.446244 22787 raft_consensus.cc:504] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.446455 22787 leader_election.cc:302] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:07.446564 22787 leader_election.cc:288] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:07.446632 22703 raft_consensus.cc:2781] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:07.446715 22703 raft_consensus.cc:686] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:07.446854 22787 ts_tablet_manager.cc:1398] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.099s user 0.001s sys 0.001s
I20240502 01:58:07.446841 22703 consensus_queue.cc:237] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.447031 22787 tablet_bootstrap.cc:492] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Bootstrap starting.
I20240502 01:58:07.448005 22787 tablet_bootstrap.cc:654] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:07.448573 21560 catalog_manager.cc:5484] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
I20240502 01:58:07.449002 22787 tablet_bootstrap.cc:492] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: No bootstrap required, opened a new log
I20240502 01:58:07.449090 22787 ts_tablet_manager.cc:1367] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:07.449462 22787 raft_consensus.cc:348] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.449559 22787 raft_consensus.cc:374] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:07.449590 22787 raft_consensus.cc:729] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Initialized, Role: FOLLOWER
I20240502 01:58:07.449716 22787 consensus_queue.cc:260] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.449798 22787 raft_consensus.cc:388] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:07.449844 22787 raft_consensus.cc:482] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:07.449893 22787 raft_consensus.cc:3037] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:07.546344 22787 raft_consensus.cc:504] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.546577 22787 leader_election.cc:302] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:07.546689 22787 leader_election.cc:288] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:07.546787 22703 raft_consensus.cc:2781] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:07.546905 22703 raft_consensus.cc:686] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Leader. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:07.546957 22787 ts_tablet_manager.cc:1398] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Time spent starting tablet: real 0.098s user 0.000s sys 0.003s
I20240502 01:58:07.547044 22703 consensus_queue.cc:237] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:07.548481 21560 catalog_manager.cc:5484] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193). New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } }
W20240502 01:58:07.674432 22795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:07.674664 22795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:07.677681 22795 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
INSERT report
rows total: 2000
time total: 56.4503 ms
time per row: 0.0282251 ms
I20240502 01:58:07.772506 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.195:42143
--local_ip_for_outbound_sockets=127.19.43.195
--tserver_master_addrs=127.19.43.254:42289
--webserver_port=39085
--webserver_interface=127.19.43.195
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:07.879870 22828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:07.880098 22828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:07.880142 22828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:07.883219 22828 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:07.883282 22828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:07.883371 22828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.195
I20240502 01:58:07.887040 22828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.195:42143
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.19.43.195
--webserver_port=39085
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.22828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.195
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:07.888010 22828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:07.889447 22828 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:07.897094 22837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:07.897095 22834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:07.897120 22835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:07.897439 22828 server_base.cc:1034] running on GCE node
I20240502 01:58:07.897826 22828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:07.898383 22828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:07.899539 22828 hybrid_clock.cc:648] HybridClock initialized: now 1714615087899524 us; error 25 us; skew 500 ppm
I20240502 01:58:07.901768 22828 webserver.cc:460] Webserver started at http://127.19.43.195:39085/ using document root <none> and password file <none>
I20240502 01:58:07.902369 22828 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:07.902443 22828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:07.906009 22828 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:58:07.907927 22843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:07.908987 22828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:07.909101 22828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f32583d958864898bc7e3567aed4334e"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:07.909545 22828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:07.932330 22828 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:07.933058 22828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:07.933233 22828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:07.934440 22828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:07.935436 22828 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:07.935488 22828 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:07.935544 22828 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:07.935571 22828 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:07.943981 22828 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.195:42143
I20240502 01:58:07.944001 22962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.195:42143 every 8 connection(s)
I20240502 01:58:07.945070 22828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 01:58:07.947518 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 22828
I20240502 01:58:07.947955 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.196:42373
--local_ip_for_outbound_sockets=127.19.43.196
--tserver_master_addrs=127.19.43.254:42289
--webserver_port=33467
--webserver_interface=127.19.43.196
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:07.954905 22963 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:07.955127 22963 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:07.955663 22963 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:07.956782 21555 ts_manager.cc:194] Re-registered known tserver with Master: f32583d958864898bc7e3567aed4334e (127.19.43.195:42143)
I20240502 01:58:07.957355 21555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.195:52113
W20240502 01:58:08.054540 22966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:08.054734 22966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:08.054793 22966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:08.057714 22966 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:08.057775 22966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:08.057854 22966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.196
I20240502 01:58:08.061596 22966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.196:42373
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.19.43.196
--webserver_port=33467
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.22966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.196
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:08.062465 22966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:08.063520 22966 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:08.071008 22972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:08.071075 22974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.071133 22966 server_base.cc:1034] running on GCE node
W20240502 01:58:08.071008 22971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.071593 22966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:08.072104 22966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:08.073239 22966 hybrid_clock.cc:648] HybridClock initialized: now 1714615088073228 us; error 30 us; skew 500 ppm
I20240502 01:58:08.074898 22966 webserver.cc:460] Webserver started at http://127.19.43.196:33467/ using document root <none> and password file <none>
I20240502 01:58:08.075416 22966 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:08.075479 22966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:08.078752 22966 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20240502 01:58:08.080550 22981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.081524 22966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:08.081625 22966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "90c9a86c10804941bfe2590985250c75"
format_stamp: "Formatted at 2024-05-02 01:58:00 on dist-test-slave-mnd3"
I20240502 01:58:08.082063 22966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:08.098392 22966 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:08.098948 22966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:08.099097 22966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:08.099581 22966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:08.100431 22966 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:08.100477 22966 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.100528 22966 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:08.100555 22966 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.108877 22966 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.196:42373
I20240502 01:58:08.108919 23098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.196:42373 every 8 connection(s)
I20240502 01:58:08.109861 22966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 01:58:08.112990 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 22966
I20240502 01:58:08.113348 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:45485
--local_ip_for_outbound_sockets=127.19.43.197
--tserver_master_addrs=127.19.43.254:42289
--webserver_port=35247
--webserver_interface=127.19.43.197
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:08.118860 23099 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:08.119107 23099 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:08.119653 23099 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:08.120668 21555 ts_manager.cc:194] Re-registered known tserver with Master: 90c9a86c10804941bfe2590985250c75 (127.19.43.196:42373)
I20240502 01:58:08.121235 21555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.196:53759
W20240502 01:58:08.219480 23102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:08.219671 23102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:08.219717 23102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:08.222707 23102 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:08.222780 23102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:08.222864 23102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:58:08.226462 23102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:45485
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=35247
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.23102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:08.227252 23102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:08.228288 23102 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:08.235639 23107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:08.235651 23108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:08.235651 23110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.235812 23102 server_base.cc:1034] running on GCE node
I20240502 01:58:08.236243 23102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:08.236769 23102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:08.237912 23102 hybrid_clock.cc:648] HybridClock initialized: now 1714615088237903 us; error 28 us; skew 500 ppm
I20240502 01:58:08.239621 23102 webserver.cc:460] Webserver started at http://127.19.43.197:35247/ using document root <none> and password file <none>
I20240502 01:58:08.240149 23102 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:08.240223 23102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:08.243722 23102 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:58:08.245632 23117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.246649 23102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:08.246769 23102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "157390243b4541548352b9d27da66cb7"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:08.247228 23102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:08.283053 23102 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:08.283731 23102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:08.283896 23102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:08.284438 23102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:08.285392 23102 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:08.285439 23102 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.285481 23102 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:08.285509 23102 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.293747 23102 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:45485
I20240502 01:58:08.293776 23233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:45485 every 8 connection(s)
I20240502 01:58:08.294806 23102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:58:08.299171 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 23102
I20240502 01:58:08.299613 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.198:35551
--local_ip_for_outbound_sockets=127.19.43.198
--tserver_master_addrs=127.19.43.254:42289
--webserver_port=38421
--webserver_interface=127.19.43.198
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:08.303676 23234 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:08.303894 23234 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:08.304371 23234 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:08.305397 21555 ts_manager.cc:194] Re-registered known tserver with Master: 157390243b4541548352b9d27da66cb7 (127.19.43.197:45485)
I20240502 01:58:08.306195 21555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:59183
W20240502 01:58:08.405058 23237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:08.405273 23237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:08.405323 23237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:08.408401 23237 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:08.408461 23237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:08.408524 23237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.198
I20240502 01:58:08.412098 23237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.198:35551
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.19.43.198
--webserver_port=38421
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.23237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.198
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:08.412976 23237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:08.414116 23237 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:08.421449 23243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:08.421483 23246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.421640 23237 server_base.cc:1034] running on GCE node
W20240502 01:58:08.421501 23242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.422081 23237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:08.422605 23237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:08.423745 23237 hybrid_clock.cc:648] HybridClock initialized: now 1714615088423736 us; error 25 us; skew 500 ppm
I20240502 01:58:08.425441 23237 webserver.cc:460] Webserver started at http://127.19.43.198:38421/ using document root <none> and password file <none>
I20240502 01:58:08.425963 23237 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:08.426065 23237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:08.429337 23237 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:08.431190 23252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.432118 23237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:08.432220 23237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "d74b9a09ea17435298a8a6f2b82281e0"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:08.432628 23237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:08.447552 23237 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:08.448169 23237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:08.448326 23237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:08.448832 23237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:08.449692 23237 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:08.449745 23237 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.449796 23237 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:08.449823 23237 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.458616 23237 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.198:35551
I20240502 01:58:08.458647 23369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.198:35551 every 8 connection(s)
I20240502 01:58:08.459705 23237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 01:58:08.465034 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 23237
I20240502 01:58:08.465449 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.199:44415
--local_ip_for_outbound_sockets=127.19.43.199
--tserver_master_addrs=127.19.43.254:42289
--webserver_port=44207
--webserver_interface=127.19.43.199
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:08.468354 23370 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:08.468557 23370 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:08.468987 23370 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:08.469964 21555 ts_manager.cc:194] Re-registered known tserver with Master: d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198:35551)
I20240502 01:58:08.470760 21555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.198:53377
W20240502 01:58:08.576409 23373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:08.576624 23373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:08.576664 23373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:08.579674 23373 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:08.579735 23373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:08.579800 23373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.199
I20240502 01:58:08.583334 23373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.199:44415
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.19.43.199
--webserver_port=44207
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.23373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.199
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:08.584167 23373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:08.585276 23373 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:08.592826 23379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:08.592828 23378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.592870 23373 server_base.cc:1034] running on GCE node
W20240502 01:58:08.592931 23381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:08.593359 23373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:08.593883 23373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:08.595028 23373 hybrid_clock.cc:648] HybridClock initialized: now 1714615088595023 us; error 39 us; skew 500 ppm
I20240502 01:58:08.596720 23373 webserver.cc:460] Webserver started at http://127.19.43.199:44207/ using document root <none> and password file <none>
I20240502 01:58:08.597221 23373 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:08.597290 23373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:08.601001 23373 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:08.602780 23388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.603719 23373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:08.603821 23373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "9e6dd740161e46459fdb27c45ae2498e"
format_stamp: "Formatted at 2024-05-02 01:58:01 on dist-test-slave-mnd3"
I20240502 01:58:08.604210 23373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:08.620638 23373 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:08.621309 23373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:08.621452 23373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:08.621960 23373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:08.622903 23373 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:08.622949 23373 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.622989 23373 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:08.623010 23373 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:08.650305 23373 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.199:44415
I20240502 01:58:08.650384 23507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.199:44415 every 8 connection(s)
I20240502 01:58:08.696697 23373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 01:58:08.702159 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 23373
I20240502 01:58:08.705848 23508 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:08.706154 23508 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:08.706753 23508 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:08.707880 21555 ts_manager.cc:194] Re-registered known tserver with Master: 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199:44415)
I20240502 01:58:08.708514 21555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.199:39807
I20240502 01:58:08.890141 21689 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:08.890470 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:08.893842 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:08.898265 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:08.904039 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:08.905855 22894 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:08.916437 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:08.959421 22963 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:09.020920 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:09.021196 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:09.024376 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:09.024888 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:09.025360 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:09.051230 22894 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:09.053830 21689 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:09.090489 21710 consensus_queue.cc:237] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
W20240502 01:58:09.095707 21642 consensus_peers.cc:485] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd -> Peer 157390243b4541548352b9d27da66cb7 (127.19.43.197:45485): Couldn't send request to peer 157390243b4541548352b9d27da66cb7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0b94526b4414b758214e8d3800d2b53. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:09.123520 23099 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:09.308130 23234 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:09.472573 23370 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:09.472630 23612 ts_tablet_manager.cc:902] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:09.475127 23612 tablet_copy_client.cc:308] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:09.478838 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet f0b94526b4414b758214e8d3800d2b53 from peer 157390243b4541548352b9d27da66cb7 ({username='slave'} at 127.19.43.197:53209)
I20240502 01:58:09.478971 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet f0b94526b4414b758214e8d3800d2b53 from peer 157390243b4541548352b9d27da66cb7 at {username='slave'} at 127.19.43.197:53209: session id = 157390243b4541548352b9d27da66cb7-f0b94526b4414b758214e8d3800d2b53
I20240502 01:58:09.480127 21731 tablet_copy_source_session.cc:205] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:09.483279 23612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b94526b4414b758214e8d3800d2b53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:09.488595 23612 tablet_copy_client.cc:783] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:09.488787 23612 tablet_copy_client.cc:647] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:09.490417 23612 tablet_copy_client.cc:523] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:09.491833 23612 tablet_bootstrap.cc:492] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Bootstrap starting.
I20240502 01:58:09.496685 23612 log.cc:824] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:09.515795 23612 tablet_bootstrap.cc:492] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:09.516268 23612 tablet_bootstrap.cc:492] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Bootstrap complete.
I20240502 01:58:09.516575 23612 ts_tablet_manager.cc:1367] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.001s
I20240502 01:58:09.518599 23612 raft_consensus.cc:348] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:09.519158 23612 raft_consensus.cc:729] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157390243b4541548352b9d27da66cb7, State: Initialized, Role: FOLLOWER
I20240502 01:58:09.519459 23612 consensus_queue.cc:260] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:09.520221 23612 ts_tablet_manager.cc:1398] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240502 01:58:09.521003 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session 157390243b4541548352b9d27da66cb7-f0b94526b4414b758214e8d3800d2b53 received from {username='slave'} at 127.19.43.197:53209
I20240502 01:58:09.521096 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session 157390243b4541548352b9d27da66cb7-f0b94526b4414b758214e8d3800d2b53 on tablet f0b94526b4414b758214e8d3800d2b53 with peer 157390243b4541548352b9d27da66cb7
I20240502 01:58:09.710927 23508 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:09.867128 23186 raft_consensus.cc:1203] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:09.868453 23620 raft_consensus.cc:2932] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 157390243b4541548352b9d27da66cb7 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:09.869062 23186 raft_consensus.cc:2932] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 157390243b4541548352b9d27da66cb7 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:09.871441 21555 catalog_manager.cc:5484] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: config changed from index -1 to 4, VOTER 157390243b4541548352b9d27da66cb7 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:09.888425 21846 consensus_queue.cc:237] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
W20240502 01:58:09.893508 21780 consensus_peers.cc:485] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe -> Peer d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198:35551): Couldn't send request to peer d74b9a09ea17435298a8a6f2b82281e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8db3067333544aaf92267389358a6fbd. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:10.319661 23648 ts_tablet_manager.cc:902] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Initiating tablet copy from peer c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:10.320372 23648 tablet_copy_client.cc:308] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:43977
I20240502 01:58:10.323832 21866 tablet_copy_service.cc:140] P c5e97283120f47719b7fcec14be91dfe: Received BeginTabletCopySession request for tablet 8db3067333544aaf92267389358a6fbd from peer d74b9a09ea17435298a8a6f2b82281e0 ({username='slave'} at 127.19.43.198:54713)
I20240502 01:58:10.323971 21866 tablet_copy_service.cc:161] P c5e97283120f47719b7fcec14be91dfe: Beginning new tablet copy session on tablet 8db3067333544aaf92267389358a6fbd from peer d74b9a09ea17435298a8a6f2b82281e0 at {username='slave'} at 127.19.43.198:54713: session id = d74b9a09ea17435298a8a6f2b82281e0-8db3067333544aaf92267389358a6fbd
I20240502 01:58:10.325119 21866 tablet_copy_source_session.cc:205] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:10.328260 23648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db3067333544aaf92267389358a6fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:10.333446 23648 tablet_copy_client.cc:783] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:10.333629 23648 tablet_copy_client.cc:647] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:10.335258 23648 tablet_copy_client.cc:523] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:10.336812 23648 tablet_bootstrap.cc:492] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Bootstrap starting.
I20240502 01:58:10.341992 23648 log.cc:824] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:10.359493 23648 tablet_bootstrap.cc:492] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:10.359874 23648 tablet_bootstrap.cc:492] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Bootstrap complete.
I20240502 01:58:10.360162 23648 ts_tablet_manager.cc:1367] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Time spent bootstrapping tablet: real 0.023s user 0.023s sys 0.000s
I20240502 01:58:10.362205 23648 raft_consensus.cc:348] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:10.362726 23648 raft_consensus.cc:729] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74b9a09ea17435298a8a6f2b82281e0, State: Initialized, Role: FOLLOWER
I20240502 01:58:10.362996 23648 consensus_queue.cc:260] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:10.363803 23648 ts_tablet_manager.cc:1398] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 01:58:10.364521 21866 tablet_copy_service.cc:342] P c5e97283120f47719b7fcec14be91dfe: Request end of tablet copy session d74b9a09ea17435298a8a6f2b82281e0-8db3067333544aaf92267389358a6fbd received from {username='slave'} at 127.19.43.198:54713
I20240502 01:58:10.364610 21866 tablet_copy_service.cc:433] P c5e97283120f47719b7fcec14be91dfe: ending tablet copy session d74b9a09ea17435298a8a6f2b82281e0-8db3067333544aaf92267389358a6fbd on tablet 8db3067333544aaf92267389358a6fbd with peer d74b9a09ea17435298a8a6f2b82281e0
I20240502 01:58:10.877463 23306 raft_consensus.cc:1203] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:10.878830 23659 raft_consensus.cc:2932] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:10.879370 23306 raft_consensus.cc:2932] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:10.881889 21554 catalog_manager.cc:5484] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe reported cstate change: config changed from index -1 to 4, VOTER d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:10.903102 21710 consensus_queue.cc:237] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
W20240502 01:58:10.907433 21641 consensus_peers.cc:485] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd -> Peer f32583d958864898bc7e3567aed4334e (127.19.43.195:42143): Couldn't send request to peer f32583d958864898bc7e3567aed4334e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8580c51fd90541c8bf70a2cee9c9ce36. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:11.514695 23687 ts_tablet_manager.cc:902] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:11.515406 23687 tablet_copy_client.cc:308] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:11.518755 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet 8580c51fd90541c8bf70a2cee9c9ce36 from peer f32583d958864898bc7e3567aed4334e ({username='slave'} at 127.19.43.195:55629)
I20240502 01:58:11.518865 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet 8580c51fd90541c8bf70a2cee9c9ce36 from peer f32583d958864898bc7e3567aed4334e at {username='slave'} at 127.19.43.195:55629: session id = f32583d958864898bc7e3567aed4334e-8580c51fd90541c8bf70a2cee9c9ce36
I20240502 01:58:11.519567 21731 tablet_copy_source_session.cc:205] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:11.522423 23687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8580c51fd90541c8bf70a2cee9c9ce36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:11.527487 23687 tablet_copy_client.cc:783] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:11.527665 23687 tablet_copy_client.cc:647] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:11.529187 23687 tablet_copy_client.cc:523] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:11.530701 23687 tablet_bootstrap.cc:492] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Bootstrap starting.
I20240502 01:58:11.535554 23687 log.cc:824] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:11.553292 23687 tablet_bootstrap.cc:492] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:11.553625 23687 tablet_bootstrap.cc:492] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Bootstrap complete.
I20240502 01:58:11.553891 23687 ts_tablet_manager.cc:1367] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Time spent bootstrapping tablet: real 0.023s user 0.020s sys 0.000s
I20240502 01:58:11.555732 23687 raft_consensus.cc:348] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:11.556253 23687 raft_consensus.cc:729] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32583d958864898bc7e3567aed4334e, State: Initialized, Role: FOLLOWER
I20240502 01:58:11.556528 23687 consensus_queue.cc:260] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:11.557310 23687 ts_tablet_manager.cc:1398] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e: Time spent starting tablet: real 0.003s user 0.007s sys 0.000s
I20240502 01:58:11.558019 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session f32583d958864898bc7e3567aed4334e-8580c51fd90541c8bf70a2cee9c9ce36 received from {username='slave'} at 127.19.43.195:55629
I20240502 01:58:11.558142 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session f32583d958864898bc7e3567aed4334e-8580c51fd90541c8bf70a2cee9c9ce36 on tablet 8580c51fd90541c8bf70a2cee9c9ce36 with peer f32583d958864898bc7e3567aed4334e
I20240502 01:58:11.953279 22916 raft_consensus.cc:1203] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:11.954443 23696 raft_consensus.cc:2932] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f32583d958864898bc7e3567aed4334e (127.19.43.195) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:11.954993 22916 raft_consensus.cc:2932] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f32583d958864898bc7e3567aed4334e (127.19.43.195) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:11.955884 23696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53718 (request call id 0) took 1053 ms. Trace:
I20240502 01:58:11.955976 23696 rpcz_store.cc:276] 0502 01:58:10.902766 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 01:58:10.902825 (+ 59us) service_pool.cc:225] Handling call
0502 01:58:10.903545 (+ 720us) log.cc:842] Serialized 260 byte log entry
0502 01:58:11.955867 (+1052322us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":46,"raft.queue_time_us":65,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":65,"threads_started":1,"wal-append.queue_time_us":124,"wal-append.run_cpu_time_us":306,"wal-append.run_wall_time_us":1000363}
I20240502 01:58:11.957617 21554 catalog_manager.cc:5484] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e reported cstate change: config changed from index -1 to 4, VOTER f32583d958864898bc7e3567aed4334e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:11.973249 21846 consensus_queue.cc:237] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
W20240502 01:58:11.977646 21779 consensus_peers.cc:485] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe -> Peer 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199:44415): Couldn't send request to peer 9e6dd740161e46459fdb27c45ae2498e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f37d05f42cca45018b116b1729755c52. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:12.365779 23723 ts_tablet_manager.cc:902] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Initiating tablet copy from peer c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:12.366462 23723 tablet_copy_client.cc:308] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:43977
I20240502 01:58:12.369904 21866 tablet_copy_service.cc:140] P c5e97283120f47719b7fcec14be91dfe: Received BeginTabletCopySession request for tablet f37d05f42cca45018b116b1729755c52 from peer 9e6dd740161e46459fdb27c45ae2498e ({username='slave'} at 127.19.43.199:51861)
I20240502 01:58:12.370043 21866 tablet_copy_service.cc:161] P c5e97283120f47719b7fcec14be91dfe: Beginning new tablet copy session on tablet f37d05f42cca45018b116b1729755c52 from peer 9e6dd740161e46459fdb27c45ae2498e at {username='slave'} at 127.19.43.199:51861: session id = 9e6dd740161e46459fdb27c45ae2498e-f37d05f42cca45018b116b1729755c52
I20240502 01:58:12.370788 21866 tablet_copy_source_session.cc:205] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:12.373579 23723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37d05f42cca45018b116b1729755c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:12.378348 23723 tablet_copy_client.cc:783] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:12.378520 23723 tablet_copy_client.cc:647] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:12.379776 23723 tablet_copy_client.cc:523] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:12.381049 23723 tablet_bootstrap.cc:492] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Bootstrap starting.
I20240502 01:58:12.386106 23723 log.cc:824] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:12.409016 23723 tablet_bootstrap.cc:492] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:12.409404 23723 tablet_bootstrap.cc:492] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Bootstrap complete.
I20240502 01:58:12.409729 23723 ts_tablet_manager.cc:1367] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Time spent bootstrapping tablet: real 0.029s user 0.026s sys 0.004s
I20240502 01:58:12.412021 23723 raft_consensus.cc:348] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:12.412732 23723 raft_consensus.cc:729] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e6dd740161e46459fdb27c45ae2498e, State: Initialized, Role: FOLLOWER
I20240502 01:58:12.413093 23723 consensus_queue.cc:260] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:12.414067 23723 ts_tablet_manager.cc:1398] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 01:58:12.414947 21866 tablet_copy_service.cc:342] P c5e97283120f47719b7fcec14be91dfe: Request end of tablet copy session 9e6dd740161e46459fdb27c45ae2498e-f37d05f42cca45018b116b1729755c52 received from {username='slave'} at 127.19.43.199:51861
I20240502 01:58:12.415035 21866 tablet_copy_service.cc:433] P c5e97283120f47719b7fcec14be91dfe: ending tablet copy session 9e6dd740161e46459fdb27c45ae2498e-f37d05f42cca45018b116b1729755c52 on tablet f37d05f42cca45018b116b1729755c52 with peer 9e6dd740161e46459fdb27c45ae2498e
I20240502 01:58:12.989290 23459 raft_consensus.cc:1203] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:12.990469 23734 raft_consensus.cc:2932] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:12.990957 23459 raft_consensus.cc:2932] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:12.991762 23734 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35038 (request call id 0) took 1018 ms. Trace:
I20240502 01:58:12.991874 23734 rpcz_store.cc:276] 0502 01:58:11.972952 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 01:58:11.973007 (+ 55us) service_pool.cc:225] Handling call
0502 01:58:11.973644 (+ 637us) log.cc:842] Serialized 260 byte log entry
0502 01:58:12.991746 (+1018102us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":42,"raft.queue_time_us":35,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":128,"thread_start_us":60,"threads_started":1,"wal-append.queue_time_us":123,"wal-append.run_cpu_time_us":298,"wal-append.run_wall_time_us":1000361}
I20240502 01:58:12.993511 21554 catalog_manager.cc:5484] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e reported cstate change: config changed from index -1 to 4, VOTER 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:13.010381 21710 consensus_queue.cc:237] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
W20240502 01:58:13.014549 21641 consensus_peers.cc:485] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd -> Peer 90c9a86c10804941bfe2590985250c75 (127.19.43.196:42373): Couldn't send request to peer 90c9a86c10804941bfe2590985250c75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b420b13c524544cd8560a93a7d5b6ee1. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:13.447806 23762 ts_tablet_manager.cc:902] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:13.448537 23762 tablet_copy_client.cc:308] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:13.452183 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet b420b13c524544cd8560a93a7d5b6ee1 from peer 90c9a86c10804941bfe2590985250c75 ({username='slave'} at 127.19.43.196:55393)
I20240502 01:58:13.452296 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet b420b13c524544cd8560a93a7d5b6ee1 from peer 90c9a86c10804941bfe2590985250c75 at {username='slave'} at 127.19.43.196:55393: session id = 90c9a86c10804941bfe2590985250c75-b420b13c524544cd8560a93a7d5b6ee1
I20240502 01:58:13.452989 21731 tablet_copy_source_session.cc:205] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:13.455758 23762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420b13c524544cd8560a93a7d5b6ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:13.460961 23762 tablet_copy_client.cc:783] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:13.461176 23762 tablet_copy_client.cc:647] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:13.462481 23762 tablet_copy_client.cc:523] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:13.463809 23762 tablet_bootstrap.cc:492] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Bootstrap starting.
I20240502 01:58:13.468389 23762 log.cc:824] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:13.487872 23762 tablet_bootstrap.cc:492] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:13.488224 23762 tablet_bootstrap.cc:492] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Bootstrap complete.
I20240502 01:58:13.488478 23762 ts_tablet_manager.cc:1367] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.000s
I20240502 01:58:13.490643 23762 raft_consensus.cc:348] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:13.491181 23762 raft_consensus.cc:729] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c9a86c10804941bfe2590985250c75, State: Initialized, Role: FOLLOWER
I20240502 01:58:13.491483 23762 consensus_queue.cc:260] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:13.492252 23762 ts_tablet_manager.cc:1398] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240502 01:58:13.492890 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session 90c9a86c10804941bfe2590985250c75-b420b13c524544cd8560a93a7d5b6ee1 received from {username='slave'} at 127.19.43.196:55393
I20240502 01:58:13.492969 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session 90c9a86c10804941bfe2590985250c75-b420b13c524544cd8560a93a7d5b6ee1 on tablet b420b13c524544cd8560a93a7d5b6ee1 with peer 90c9a86c10804941bfe2590985250c75
I20240502 01:58:13.862203 23051 raft_consensus.cc:1203] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:13.863327 23771 raft_consensus.cc:2932] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 90c9a86c10804941bfe2590985250c75 (127.19.43.196) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:13.863890 23051 raft_consensus.cc:2932] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 90c9a86c10804941bfe2590985250c75 (127.19.43.196) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:13.865622 21555 catalog_manager.cc:5484] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: config changed from index -1 to 4, VOTER 90c9a86c10804941bfe2590985250c75 (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:13.882370 21846 consensus_queue.cc:237] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
W20240502 01:58:13.886423 21779 consensus_peers.cc:485] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe -> Peer 157390243b4541548352b9d27da66cb7 (127.19.43.197:45485): Couldn't send request to peer 157390243b4541548352b9d27da66cb7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ac53985ac124e23ac10c86b2b048aa8. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:14.400544 23796 ts_tablet_manager.cc:902] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: Initiating tablet copy from peer c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:14.400964 23796 tablet_copy_client.cc:308] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:43977
I20240502 01:58:14.404304 21866 tablet_copy_service.cc:140] P c5e97283120f47719b7fcec14be91dfe: Received BeginTabletCopySession request for tablet 0ac53985ac124e23ac10c86b2b048aa8 from peer 157390243b4541548352b9d27da66cb7 ({username='slave'} at 127.19.43.197:35361)
I20240502 01:58:14.404399 21866 tablet_copy_service.cc:161] P c5e97283120f47719b7fcec14be91dfe: Beginning new tablet copy session on tablet 0ac53985ac124e23ac10c86b2b048aa8 from peer 157390243b4541548352b9d27da66cb7 at {username='slave'} at 127.19.43.197:35361: session id = 157390243b4541548352b9d27da66cb7-0ac53985ac124e23ac10c86b2b048aa8
I20240502 01:58:14.405095 21866 tablet_copy_source_session.cc:205] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:14.405854 23796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac53985ac124e23ac10c86b2b048aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:14.407482 23796 tablet_copy_client.cc:783] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:14.407640 23796 tablet_copy_client.cc:647] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:14.408619 23796 tablet_copy_client.cc:523] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:14.409570 23796 tablet_bootstrap.cc:492] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: Bootstrap starting.
I20240502 01:58:14.425171 23796 tablet_bootstrap.cc:492] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:14.425374 23796 tablet_bootstrap.cc:492] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: Bootstrap complete.
I20240502 01:58:14.425504 23796 ts_tablet_manager.cc:1367] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20240502 01:58:14.425971 23796 raft_consensus.cc:348] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:14.426175 23796 raft_consensus.cc:729] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157390243b4541548352b9d27da66cb7, State: Initialized, Role: FOLLOWER
I20240502 01:58:14.426272 23796 consensus_queue.cc:260] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:14.426535 23796 ts_tablet_manager.cc:1398] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 01:58:14.426980 21866 tablet_copy_service.cc:342] P c5e97283120f47719b7fcec14be91dfe: Request end of tablet copy session 157390243b4541548352b9d27da66cb7-0ac53985ac124e23ac10c86b2b048aa8 received from {username='slave'} at 127.19.43.197:35361
I20240502 01:58:14.427057 21866 tablet_copy_service.cc:433] P c5e97283120f47719b7fcec14be91dfe: ending tablet copy session 157390243b4541548352b9d27da66cb7-0ac53985ac124e23ac10c86b2b048aa8 on tablet 0ac53985ac124e23ac10c86b2b048aa8 with peer 157390243b4541548352b9d27da66cb7
I20240502 01:58:14.964926 23186 raft_consensus.cc:1203] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:14.965750 23806 raft_consensus.cc:2932] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 157390243b4541548352b9d27da66cb7 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:14.966203 23186 raft_consensus.cc:2932] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 157390243b4541548352b9d27da66cb7 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:14.967130 23806 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35058 (request call id 0) took 1085 ms. Trace:
I20240502 01:58:14.967262 23806 rpcz_store.cc:276] 0502 01:58:13.882078 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 01:58:13.882119 (+ 41us) service_pool.cc:225] Handling call
0502 01:58:13.882766 (+ 647us) log.cc:842] Serialized 260 byte log entry
0502 01:58:14.967113 (+1084347us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":43,"raft.queue_time_us":35,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"thread_start_us":59,"threads_started":1,"wal-append.queue_time_us":117,"wal-append.run_cpu_time_us":280,"wal-append.run_wall_time_us":1000339}
I20240502 01:58:14.968358 21554 catalog_manager.cc:5484] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 reported cstate change: config changed from index -1 to 4, VOTER 157390243b4541548352b9d27da66cb7 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:14.984436 21710 consensus_queue.cc:237] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
W20240502 01:58:14.988581 21643 consensus_peers.cc:485] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd -> Peer d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198:35551): Couldn't send request to peer d74b9a09ea17435298a8a6f2b82281e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c03ce899e48c4447b56b1353c3f1d818. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:15.520990 23835 ts_tablet_manager.cc:902] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:15.521407 23835 tablet_copy_client.cc:308] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:15.524715 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet c03ce899e48c4447b56b1353c3f1d818 from peer d74b9a09ea17435298a8a6f2b82281e0 ({username='slave'} at 127.19.43.198:57167)
I20240502 01:58:15.524807 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet c03ce899e48c4447b56b1353c3f1d818 from peer d74b9a09ea17435298a8a6f2b82281e0 at {username='slave'} at 127.19.43.198:57167: session id = d74b9a09ea17435298a8a6f2b82281e0-c03ce899e48c4447b56b1353c3f1d818
I20240502 01:58:15.525461 21731 tablet_copy_source_session.cc:205] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:15.526263 23835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03ce899e48c4447b56b1353c3f1d818. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:15.527957 23835 tablet_copy_client.cc:783] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:15.528100 23835 tablet_copy_client.cc:647] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:15.529171 23835 tablet_copy_client.cc:523] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:15.530203 23835 tablet_bootstrap.cc:492] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: Bootstrap starting.
I20240502 01:58:15.547552 23835 tablet_bootstrap.cc:492] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:15.547799 23835 tablet_bootstrap.cc:492] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: Bootstrap complete.
I20240502 01:58:15.547962 23835 ts_tablet_manager.cc:1367] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20240502 01:58:15.548422 23835 raft_consensus.cc:348] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:15.548622 23835 raft_consensus.cc:729] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74b9a09ea17435298a8a6f2b82281e0, State: Initialized, Role: FOLLOWER
I20240502 01:58:15.548727 23835 consensus_queue.cc:260] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:15.549027 23835 ts_tablet_manager.cc:1398] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:58:15.550143 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session d74b9a09ea17435298a8a6f2b82281e0-c03ce899e48c4447b56b1353c3f1d818 received from {username='slave'} at 127.19.43.198:57167
I20240502 01:58:15.550230 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session d74b9a09ea17435298a8a6f2b82281e0-c03ce899e48c4447b56b1353c3f1d818 on tablet c03ce899e48c4447b56b1353c3f1d818 with peer d74b9a09ea17435298a8a6f2b82281e0
I20240502 01:58:16.011646 23306 raft_consensus.cc:1203] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:16.012437 23844 raft_consensus.cc:2932] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:16.012887 23306 raft_consensus.cc:2932] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:16.013809 23844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53738 (request call id 0) took 1029 ms. Trace:
I20240502 01:58:16.013867 23844 rpcz_store.cc:276] 0502 01:58:14.984158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 01:58:14.984208 (+ 50us) service_pool.cc:225] Handling call
0502 01:58:14.984835 (+ 627us) log.cc:842] Serialized 260 byte log entry
0502 01:58:16.013795 (+1028960us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":43,"raft.queue_time_us":33,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":58,"threads_started":1,"wal-append.queue_time_us":111,"wal-append.run_cpu_time_us":294,"wal-append.run_wall_time_us":1000300}
I20240502 01:58:16.015583 21554 catalog_manager.cc:5484] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: config changed from index -1 to 4, VOTER d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:16.031564 21710 consensus_queue.cc:237] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
W20240502 01:58:16.032737 21641 consensus_peers.cc:485] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd -> Peer f32583d958864898bc7e3567aed4334e (127.19.43.195:42143): Couldn't send request to peer f32583d958864898bc7e3567aed4334e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f2f2a1efb8da4f32ad8a2e74610892dd. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:16.567433 23868 ts_tablet_manager.cc:902] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:16.567857 23868 tablet_copy_client.cc:308] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:16.568318 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet f2f2a1efb8da4f32ad8a2e74610892dd from peer f32583d958864898bc7e3567aed4334e ({username='slave'} at 127.19.43.195:55629)
I20240502 01:58:16.568418 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet f2f2a1efb8da4f32ad8a2e74610892dd from peer f32583d958864898bc7e3567aed4334e at {username='slave'} at 127.19.43.195:55629: session id = f32583d958864898bc7e3567aed4334e-f2f2a1efb8da4f32ad8a2e74610892dd
I20240502 01:58:16.569090 21731 tablet_copy_source_session.cc:205] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:16.569872 23868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2a1efb8da4f32ad8a2e74610892dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:16.571583 23868 tablet_copy_client.cc:783] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:16.571725 23868 tablet_copy_client.cc:647] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:16.572726 23868 tablet_copy_client.cc:523] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:16.573637 23868 tablet_bootstrap.cc:492] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: Bootstrap starting.
I20240502 01:58:16.590157 23868 tablet_bootstrap.cc:492] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:16.590370 23868 tablet_bootstrap.cc:492] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: Bootstrap complete.
I20240502 01:58:16.590503 23868 ts_tablet_manager.cc:1367] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.000s
I20240502 01:58:16.590924 23868 raft_consensus.cc:348] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:16.591104 23868 raft_consensus.cc:729] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32583d958864898bc7e3567aed4334e, State: Initialized, Role: FOLLOWER
I20240502 01:58:16.591193 23868 consensus_queue.cc:260] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:16.591481 23868 ts_tablet_manager.cc:1398] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:16.591895 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session f32583d958864898bc7e3567aed4334e-f2f2a1efb8da4f32ad8a2e74610892dd received from {username='slave'} at 127.19.43.195:55629
I20240502 01:58:16.591961 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session f32583d958864898bc7e3567aed4334e-f2f2a1efb8da4f32ad8a2e74610892dd on tablet f2f2a1efb8da4f32ad8a2e74610892dd with peer f32583d958864898bc7e3567aed4334e
I20240502 01:58:17.008010 22916 raft_consensus.cc:1203] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:17.008925 23876 raft_consensus.cc:2932] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f32583d958864898bc7e3567aed4334e (127.19.43.195) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:17.009696 22916 raft_consensus.cc:2932] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f32583d958864898bc7e3567aed4334e (127.19.43.195) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:17.011672 21554 catalog_manager.cc:5484] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: config changed from index -1 to 4, VOTER f32583d958864898bc7e3567aed4334e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:17.029050 21846 consensus_queue.cc:237] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
W20240502 01:58:17.033367 21778 consensus_peers.cc:485] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe -> Peer 90c9a86c10804941bfe2590985250c75 (127.19.43.196:42373): Couldn't send request to peer 90c9a86c10804941bfe2590985250c75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a3ca0726e154d74aa3da3afc065c827. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:17.604629 23901 ts_tablet_manager.cc:902] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: Initiating tablet copy from peer c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:17.605077 23901 tablet_copy_client.cc:308] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:43977
I20240502 01:58:17.608412 21866 tablet_copy_service.cc:140] P c5e97283120f47719b7fcec14be91dfe: Received BeginTabletCopySession request for tablet 1a3ca0726e154d74aa3da3afc065c827 from peer 90c9a86c10804941bfe2590985250c75 ({username='slave'} at 127.19.43.196:45887)
I20240502 01:58:17.608508 21866 tablet_copy_service.cc:161] P c5e97283120f47719b7fcec14be91dfe: Beginning new tablet copy session on tablet 1a3ca0726e154d74aa3da3afc065c827 from peer 90c9a86c10804941bfe2590985250c75 at {username='slave'} at 127.19.43.196:45887: session id = 90c9a86c10804941bfe2590985250c75-1a3ca0726e154d74aa3da3afc065c827
I20240502 01:58:17.609282 21866 tablet_copy_source_session.cc:205] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:17.610096 23901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a3ca0726e154d74aa3da3afc065c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:17.611749 23901 tablet_copy_client.cc:783] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:17.611902 23901 tablet_copy_client.cc:647] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:17.612910 23901 tablet_copy_client.cc:523] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:17.613921 23901 tablet_bootstrap.cc:492] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: Bootstrap starting.
I20240502 01:58:17.631278 23901 tablet_bootstrap.cc:492] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:17.631480 23901 tablet_bootstrap.cc:492] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: Bootstrap complete.
I20240502 01:58:17.631601 23901 ts_tablet_manager.cc:1367] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20240502 01:58:17.632062 23901 raft_consensus.cc:348] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:17.632231 23901 raft_consensus.cc:729] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c9a86c10804941bfe2590985250c75, State: Initialized, Role: FOLLOWER
I20240502 01:58:17.632304 23901 consensus_queue.cc:260] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:17.632567 23901 ts_tablet_manager.cc:1398] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:58:17.632973 21866 tablet_copy_service.cc:342] P c5e97283120f47719b7fcec14be91dfe: Request end of tablet copy session 90c9a86c10804941bfe2590985250c75-1a3ca0726e154d74aa3da3afc065c827 received from {username='slave'} at 127.19.43.196:45887
I20240502 01:58:17.633047 21866 tablet_copy_service.cc:433] P c5e97283120f47719b7fcec14be91dfe: ending tablet copy session 90c9a86c10804941bfe2590985250c75-1a3ca0726e154d74aa3da3afc065c827 on tablet 1a3ca0726e154d74aa3da3afc065c827 with peer 90c9a86c10804941bfe2590985250c75
I20240502 01:58:17.982622 23051 raft_consensus.cc:1203] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:17.983557 23910 raft_consensus.cc:2932] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 90c9a86c10804941bfe2590985250c75 (127.19.43.196) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:17.984088 23051 raft_consensus.cc:2932] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 90c9a86c10804941bfe2590985250c75 (127.19.43.196) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:17.986498 21555 catalog_manager.cc:5484] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe reported cstate change: config changed from index -1 to 4, VOTER 90c9a86c10804941bfe2590985250c75 (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:18.040978 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.042897 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.044099 21689 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:18.046624 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.048878 22894 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.048882 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.074754 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.094473 21846 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1a3ca0726e154d74aa3da3afc065c827"
dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35116
I20240502 01:58:18.094631 21846 raft_consensus.cc:595] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:18.143033 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.143769 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.144390 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.144806 21689 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:18.155146 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.169912 22894 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.170578 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.246974 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.247740 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.249042 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.260862 22894 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.262413 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.263334 21689 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:18.271417 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.296362 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f2f2a1efb8da4f32ad8a2e74610892dd"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53792
I20240502 01:58:18.296545 21710 raft_consensus.cc:595] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:18.344007 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.346539 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.347695 21689 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:18.358929 22894 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.360770 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.361178 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.361893 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.423657 23876 raft_consensus.cc:979] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower f32583d958864898bc7e3567aed4334e to start an election
I20240502 01:58:18.423761 23876 raft_consensus.cc:1067] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer f32583d958864898bc7e3567aed4334e to start an election
I20240502 01:58:18.424324 22916 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2a1efb8da4f32ad8a2e74610892dd"
dest_uuid: "f32583d958864898bc7e3567aed4334e"
from {username='slave'} at 127.19.43.193:44247
I20240502 01:58:18.424494 22916 raft_consensus.cc:482] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:18.424600 22916 raft_consensus.cc:3037] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:18.425700 22916 raft_consensus.cc:504] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:18.426717 22916 leader_election.cc:288] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:18.427074 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2a1efb8da4f32ad8a2e74610892dd" candidate_uuid: "f32583d958864898bc7e3567aed4334e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:18.427238 21710 raft_consensus.cc:3032] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:18.427270 21710 raft_consensus.cc:729] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:18.427417 21710 consensus_queue.cc:260] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:18.427668 21710 raft_consensus.cc:3037] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:18.428436 21710 raft_consensus.cc:2454] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f32583d958864898bc7e3567aed4334e in term 2.
I20240502 01:58:18.428820 22847 leader_election.cc:302] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd, f32583d958864898bc7e3567aed4334e; no voters:
I20240502 01:58:18.429124 24106 raft_consensus.cc:2781] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:18.429303 24106 raft_consensus.cc:686] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 2 LEADER]: Becoming Leader. State: Replica: f32583d958864898bc7e3567aed4334e, State: Running, Role: LEADER
I20240502 01:58:18.429595 24106 consensus_queue.cc:237] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:18.431982 21555 catalog_manager.cc:5484] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to f32583d958864898bc7e3567aed4334e (127.19.43.195). New cstate: current_term: 2 leader_uuid: "f32583d958864898bc7e3567aed4334e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:18.446368 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.446466 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.450618 21689 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:18.454908 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.455632 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.464553 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.481593 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.505314 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c03ce899e48c4447b56b1353c3f1d818"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53828
I20240502 01:58:18.505434 21710 raft_consensus.cc:595] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:18.553225 21689 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:18.553838 23301 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.553881 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.554059 21823 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.559675 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.569811 23030 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.570873 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.583154 23910 raft_consensus.cc:979] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: : Instructing follower 90c9a86c10804941bfe2590985250c75 to start an election
I20240502 01:58:18.583230 23910 raft_consensus.cc:1067] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Signalling peer 90c9a86c10804941bfe2590985250c75 to start an election
I20240502 01:58:18.583904 23051 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1a3ca0726e154d74aa3da3afc065c827"
dest_uuid: "90c9a86c10804941bfe2590985250c75"
from {username='slave'} at 127.19.43.194:41005
I20240502 01:58:18.584071 23051 raft_consensus.cc:482] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:18.584159 23051 raft_consensus.cc:3037] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:18.585148 23051 raft_consensus.cc:504] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:18.586108 23051 leader_election.cc:288] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [CANDIDATE]: Term 2 election: Requested vote from peers c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:18.586535 21846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3ca0726e154d74aa3da3afc065c827" candidate_uuid: "90c9a86c10804941bfe2590985250c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
I20240502 01:58:18.586688 21846 raft_consensus.cc:3032] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:18.586746 21846 raft_consensus.cc:729] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:18.586889 21846 consensus_queue.cc:260] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:18.587098 21846 raft_consensus.cc:3037] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:18.588085 21846 raft_consensus.cc:2454] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c9a86c10804941bfe2590985250c75 in term 2.
I20240502 01:58:18.588495 22983 leader_election.cc:302] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 90c9a86c10804941bfe2590985250c75, c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:18.588820 24159 raft_consensus.cc:2781] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:18.589013 24159 raft_consensus.cc:686] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 2 LEADER]: Becoming Leader. State: Replica: 90c9a86c10804941bfe2590985250c75, State: Running, Role: LEADER
I20240502 01:58:18.589334 24159 consensus_queue.cc:237] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:18.591493 21554 catalog_manager.cc:5484] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 reported cstate change: term changed from 1 to 2, leader changed from c5e97283120f47719b7fcec14be91dfe (127.19.43.194) to 90c9a86c10804941bfe2590985250c75 (127.19.43.196). New cstate: current_term: 2 leader_uuid: "90c9a86c10804941bfe2590985250c75" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:18.639636 24099 raft_consensus.cc:979] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower d74b9a09ea17435298a8a6f2b82281e0 to start an election
I20240502 01:58:18.639737 24099 raft_consensus.cc:1067] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer d74b9a09ea17435298a8a6f2b82281e0 to start an election
I20240502 01:58:18.640379 23306 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c03ce899e48c4447b56b1353c3f1d818"
dest_uuid: "d74b9a09ea17435298a8a6f2b82281e0"
from {username='slave'} at 127.19.43.193:40441
I20240502 01:58:18.640576 23306 raft_consensus.cc:482] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:18.640695 23306 raft_consensus.cc:3037] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:18.641687 23306 raft_consensus.cc:504] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:18.642567 23306 leader_election.cc:288] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:18.642938 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03ce899e48c4447b56b1353c3f1d818" candidate_uuid: "d74b9a09ea17435298a8a6f2b82281e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:18.643047 21710 raft_consensus.cc:3032] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:18.643090 21710 raft_consensus.cc:729] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:18.643221 21710 consensus_queue.cc:260] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:18.643361 21710 raft_consensus.cc:3037] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:18.644351 21710 raft_consensus.cc:2454] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74b9a09ea17435298a8a6f2b82281e0 in term 2.
I20240502 01:58:18.646100 23257 leader_election.cc:302] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd, d74b9a09ea17435298a8a6f2b82281e0; no voters:
I20240502 01:58:18.646907 24199 raft_consensus.cc:2781] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:18.647082 24199 raft_consensus.cc:686] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 2 LEADER]: Becoming Leader. State: Replica: d74b9a09ea17435298a8a6f2b82281e0, State: Running, Role: LEADER
I20240502 01:58:18.647429 24199 consensus_queue.cc:237] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:18.652015 21559 catalog_manager.cc:5484] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198). New cstate: current_term: 2 leader_uuid: "d74b9a09ea17435298a8a6f2b82281e0" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:18.655576 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.657021 23301 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.660128 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.671713 21689 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.672641 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.673753 21823 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:18.693534 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.712208 21846 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8db3067333544aaf92267389358a6fbd"
dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35214
I20240502 01:58:18.712338 21846 raft_consensus.cc:595] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:18.757761 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.759618 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.762429 21689 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.762521 23301 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.763904 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.772127 21823 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:18.775329 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.823347 21710 raft_consensus.cc:1261] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer f32583d958864898bc7e3567aed4334e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:18.823952 24106 consensus_queue.cc:1035] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:18.854149 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.855162 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.860507 21689 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.860831 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.866151 21823 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:18.866595 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.878604 23301 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.903367 21846 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0ac53985ac124e23ac10c86b2b048aa8"
dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56820
I20240502 01:58:18.903497 21846 raft_consensus.cc:595] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:18.948836 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.950989 21823 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:18.954221 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:18.955730 21689 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:18.966269 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.973007 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:18.976531 23301 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.046619 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.047819 21689 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:19.048768 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.050238 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.053493 23301 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.065455 22894 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.073596 21823 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:19.092657 24356 raft_consensus.cc:979] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: : Instructing follower d74b9a09ea17435298a8a6f2b82281e0 to start an election
I20240502 01:58:19.092795 23910 raft_consensus.cc:1067] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Signalling peer d74b9a09ea17435298a8a6f2b82281e0 to start an election
I20240502 01:58:19.093575 23306 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8db3067333544aaf92267389358a6fbd"
dest_uuid: "d74b9a09ea17435298a8a6f2b82281e0"
from {username='slave'} at 127.19.43.194:56953
I20240502 01:58:19.093713 23306 raft_consensus.cc:482] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:19.093773 23306 raft_consensus.cc:3037] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.094399 21710 raft_consensus.cc:1261] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer d74b9a09ea17435298a8a6f2b82281e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.094766 23306 raft_consensus.cc:504] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:19.094839 24199 consensus_queue.cc:1035] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:19.095099 23306 leader_election.cc:288] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [CANDIDATE]: Term 2 election: Requested vote from peers c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:19.095407 21846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db3067333544aaf92267389358a6fbd" candidate_uuid: "d74b9a09ea17435298a8a6f2b82281e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
I20240502 01:58:19.095499 21846 raft_consensus.cc:3032] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:19.095525 21846 raft_consensus.cc:729] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:19.095649 21846 consensus_queue.cc:260] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:19.095806 21846 raft_consensus.cc:3037] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.097193 21846 raft_consensus.cc:2454] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74b9a09ea17435298a8a6f2b82281e0 in term 2.
I20240502 01:58:19.097486 23254 leader_election.cc:302] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: c5e97283120f47719b7fcec14be91dfe, d74b9a09ea17435298a8a6f2b82281e0; no voters:
I20240502 01:58:19.097697 24200 raft_consensus.cc:2781] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:19.097792 24200 raft_consensus.cc:686] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 2 LEADER]: Becoming Leader. State: Replica: d74b9a09ea17435298a8a6f2b82281e0, State: Running, Role: LEADER
I20240502 01:58:19.097910 24200 consensus_queue.cc:237] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:19.099004 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8580c51fd90541c8bf70a2cee9c9ce36"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:33718
I20240502 01:58:19.099108 21710 raft_consensus.cc:595] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:19.099687 21552 catalog_manager.cc:5484] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 reported cstate change: term changed from 1 to 2, leader changed from c5e97283120f47719b7fcec14be91dfe (127.19.43.194) to d74b9a09ea17435298a8a6f2b82281e0 (127.19.43.198). New cstate: current_term: 2 leader_uuid: "d74b9a09ea17435298a8a6f2b82281e0" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:19.110144 24099 raft_consensus.cc:979] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower f32583d958864898bc7e3567aed4334e to start an election
I20240502 01:58:19.110239 24099 raft_consensus.cc:1067] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer f32583d958864898bc7e3567aed4334e to start an election
I20240502 01:58:19.110657 22916 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8580c51fd90541c8bf70a2cee9c9ce36"
dest_uuid: "f32583d958864898bc7e3567aed4334e"
from {username='slave'} at 127.19.43.193:44247
I20240502 01:58:19.110788 22916 raft_consensus.cc:482] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:19.110843 22916 raft_consensus.cc:3037] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.111676 22916 raft_consensus.cc:504] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:19.111968 22916 leader_election.cc:288] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:19.112355 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8580c51fd90541c8bf70a2cee9c9ce36" candidate_uuid: "f32583d958864898bc7e3567aed4334e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:19.112455 21710 raft_consensus.cc:3032] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:19.112491 21710 raft_consensus.cc:729] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:19.112603 21710 consensus_queue.cc:260] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:19.112725 21710 raft_consensus.cc:3037] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.113564 21710 raft_consensus.cc:2454] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f32583d958864898bc7e3567aed4334e in term 2.
I20240502 01:58:19.113826 22847 leader_election.cc:302] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd, f32583d958864898bc7e3567aed4334e; no voters:
I20240502 01:58:19.113993 24106 raft_consensus.cc:2781] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:19.114127 24106 raft_consensus.cc:686] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 2 LEADER]: Becoming Leader. State: Replica: f32583d958864898bc7e3567aed4334e, State: Running, Role: LEADER
I20240502 01:58:19.114298 24106 consensus_queue.cc:237] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:19.115576 21552 catalog_manager.cc:5484] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to f32583d958864898bc7e3567aed4334e (127.19.43.195). New cstate: current_term: 2 leader_uuid: "f32583d958864898bc7e3567aed4334e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:19.153199 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.154256 21689 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:19.156663 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.158423 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.167922 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.175904 21823 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:19.184114 21846 raft_consensus.cc:1261] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Refusing update from remote peer 90c9a86c10804941bfe2590985250c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.184749 24398 consensus_queue.cc:1035] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:19.191783 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.260288 21689 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:19.260752 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.261179 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.276914 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.276914 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.279773 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.287842 21823 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:19.312347 21846 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f37d05f42cca45018b116b1729755c52"
dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56886
I20240502 01:58:19.312465 21846 raft_consensus.cc:595] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:19.356899 23166 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.358886 21689 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:19.360586 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.366176 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.366891 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.374498 21823 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:19.375566 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.398234 24356 raft_consensus.cc:979] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: : Instructing follower 157390243b4541548352b9d27da66cb7 to start an election
I20240502 01:58:19.398336 24356 raft_consensus.cc:1067] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Signalling peer 157390243b4541548352b9d27da66cb7 to start an election
I20240502 01:58:19.398749 23186 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ac53985ac124e23ac10c86b2b048aa8"
dest_uuid: "157390243b4541548352b9d27da66cb7"
from {username='slave'} at 127.19.43.194:34959
I20240502 01:58:19.398928 23186 raft_consensus.cc:482] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:19.399040 23186 raft_consensus.cc:3037] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.399995 23186 raft_consensus.cc:504] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:19.400844 23186 leader_election.cc:288] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [CANDIDATE]: Term 2 election: Requested vote from peers c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:19.401160 21846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac53985ac124e23ac10c86b2b048aa8" candidate_uuid: "157390243b4541548352b9d27da66cb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
I20240502 01:58:19.401257 21846 raft_consensus.cc:3032] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:19.401286 21846 raft_consensus.cc:729] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:19.401412 21846 consensus_queue.cc:260] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:19.401556 21846 raft_consensus.cc:3037] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.402315 21846 raft_consensus.cc:2454] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 157390243b4541548352b9d27da66cb7 in term 2.
I20240502 01:58:19.402655 23119 leader_election.cc:302] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 157390243b4541548352b9d27da66cb7, c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:19.403028 24481 raft_consensus.cc:2781] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:19.403184 24481 raft_consensus.cc:686] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 2 LEADER]: Becoming Leader. State: Replica: 157390243b4541548352b9d27da66cb7, State: Running, Role: LEADER
I20240502 01:58:19.403602 24481 consensus_queue.cc:237] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:19.405702 21553 catalog_manager.cc:5484] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 reported cstate change: term changed from 1 to 2, leader changed from c5e97283120f47719b7fcec14be91dfe (127.19.43.194) to 157390243b4541548352b9d27da66cb7 (127.19.43.197). New cstate: current_term: 2 leader_uuid: "157390243b4541548352b9d27da66cb7" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:19.454288 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.455790 23166 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.455859 21689 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:19.458405 23439 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:19.462200 21823 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:19.485162 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.495446 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.515564 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f0b94526b4414b758214e8d3800d2b53"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:33764
I20240502 01:58:19.515683 21710 raft_consensus.cc:595] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:19.532083 24356 raft_consensus.cc:979] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: : Instructing follower 9e6dd740161e46459fdb27c45ae2498e to start an election
I20240502 01:58:19.532172 24356 raft_consensus.cc:1067] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Signalling peer 9e6dd740161e46459fdb27c45ae2498e to start an election
I20240502 01:58:19.532627 23459 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f37d05f42cca45018b116b1729755c52"
dest_uuid: "9e6dd740161e46459fdb27c45ae2498e"
from {username='slave'} at 127.19.43.194:51933
I20240502 01:58:19.532788 23459 raft_consensus.cc:482] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:19.532884 23459 raft_consensus.cc:3037] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.533774 23459 raft_consensus.cc:504] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:19.534691 23459 leader_election.cc:288] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [CANDIDATE]: Term 2 election: Requested vote from peers c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:19.535003 21846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37d05f42cca45018b116b1729755c52" candidate_uuid: "9e6dd740161e46459fdb27c45ae2498e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
I20240502 01:58:19.535108 21846 raft_consensus.cc:3032] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:19.535154 21846 raft_consensus.cc:729] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:19.535296 21846 consensus_queue.cc:260] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:19.535459 21846 raft_consensus.cc:3037] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.536336 21846 raft_consensus.cc:2454] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e6dd740161e46459fdb27c45ae2498e in term 2.
I20240502 01:58:19.536639 23390 leader_election.cc:302] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 9e6dd740161e46459fdb27c45ae2498e, c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:19.536942 24539 raft_consensus.cc:2781] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:19.537087 24539 raft_consensus.cc:686] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 2 LEADER]: Becoming Leader. State: Replica: 9e6dd740161e46459fdb27c45ae2498e, State: Running, Role: LEADER
I20240502 01:58:19.537369 24539 consensus_queue.cc:237] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:19.539642 21554 catalog_manager.cc:5484] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e reported cstate change: term changed from 1 to 2, leader changed from c5e97283120f47719b7fcec14be91dfe (127.19.43.194) to 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199). New cstate: current_term: 2 leader_uuid: "9e6dd740161e46459fdb27c45ae2498e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:19.563100 23166 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.566485 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.567430 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.567885 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:19.568737 21689 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:19.569360 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.584080 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.590699 23876 raft_consensus.cc:979] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower 157390243b4541548352b9d27da66cb7 to start an election
I20240502 01:58:19.590785 23876 raft_consensus.cc:1067] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer 157390243b4541548352b9d27da66cb7 to start an election
I20240502 01:58:19.591187 23186 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f0b94526b4414b758214e8d3800d2b53"
dest_uuid: "157390243b4541548352b9d27da66cb7"
from {username='slave'} at 127.19.43.193:33595
I20240502 01:58:19.591325 23186 raft_consensus.cc:482] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:19.591384 23186 raft_consensus.cc:3037] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.592301 23186 raft_consensus.cc:504] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:19.592633 23186 leader_election.cc:288] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:19.593101 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b94526b4414b758214e8d3800d2b53" candidate_uuid: "157390243b4541548352b9d27da66cb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:19.593201 21710 raft_consensus.cc:3032] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:19.593232 21710 raft_consensus.cc:729] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:19.593353 21710 consensus_queue.cc:260] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:19.593475 21710 raft_consensus.cc:3037] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.594350 21710 raft_consensus.cc:2454] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 157390243b4541548352b9d27da66cb7 in term 2.
I20240502 01:58:19.594700 23121 leader_election.cc:302] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 157390243b4541548352b9d27da66cb7, 1ab91a4d0007452a85f25102557ed3dd; no voters:
I20240502 01:58:19.594914 24481 raft_consensus.cc:2781] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:19.594990 24481 raft_consensus.cc:686] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 2 LEADER]: Becoming Leader. State: Replica: 157390243b4541548352b9d27da66cb7, State: Running, Role: LEADER
I20240502 01:58:19.595098 24481 consensus_queue.cc:237] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:19.596836 21554 catalog_manager.cc:5484] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to 157390243b4541548352b9d27da66cb7 (127.19.43.197). New cstate: current_term: 2 leader_uuid: "157390243b4541548352b9d27da66cb7" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:19.604722 21710 raft_consensus.cc:1261] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer f32583d958864898bc7e3567aed4334e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.605125 24106 consensus_queue.cc:1035] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:19.606082 21846 raft_consensus.cc:1261] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Refusing update from remote peer d74b9a09ea17435298a8a6f2b82281e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.606696 24200 consensus_queue.cc:1035] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:19.675052 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.679638 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:19.680155 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.682336 21689 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:19.686683 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.696242 23030 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.696946 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.728634 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b420b13c524544cd8560a93a7d5b6ee1"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:33798
I20240502 01:58:19.728762 21710 raft_consensus.cc:595] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:19.769246 23876 raft_consensus.cc:979] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower 90c9a86c10804941bfe2590985250c75 to start an election
I20240502 01:58:19.769352 23876 raft_consensus.cc:1067] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer 90c9a86c10804941bfe2590985250c75 to start an election
I20240502 01:58:19.770377 23051 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b420b13c524544cd8560a93a7d5b6ee1"
dest_uuid: "90c9a86c10804941bfe2590985250c75"
from {username='slave'} at 127.19.43.193:44905
I20240502 01:58:19.770514 23051 raft_consensus.cc:482] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:19.770581 23051 raft_consensus.cc:3037] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.772998 23051 raft_consensus.cc:504] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:19.773386 23051 leader_election.cc:288] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:19.773835 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420b13c524544cd8560a93a7d5b6ee1" candidate_uuid: "90c9a86c10804941bfe2590985250c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:19.773945 21710 raft_consensus.cc:3032] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:19.773981 21710 raft_consensus.cc:729] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:19.774138 21710 consensus_queue.cc:260] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:19.774278 21710 raft_consensus.cc:3037] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:19.775233 21710 raft_consensus.cc:2454] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c9a86c10804941bfe2590985250c75 in term 2.
I20240502 01:58:19.776271 22985 leader_election.cc:302] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd, 90c9a86c10804941bfe2590985250c75; no voters:
I20240502 01:58:19.776783 24620 raft_consensus.cc:2781] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:19.776867 24620 raft_consensus.cc:686] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 2 LEADER]: Becoming Leader. State: Replica: 90c9a86c10804941bfe2590985250c75, State: Running, Role: LEADER
I20240502 01:58:19.776994 24620 consensus_queue.cc:237] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:19.778736 21559 catalog_manager.cc:5484] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to 90c9a86c10804941bfe2590985250c75 (127.19.43.196). New cstate: current_term: 2 leader_uuid: "90c9a86c10804941bfe2590985250c75" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:19.780661 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:19.781340 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:19.782305 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:19.782653 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.783638 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.799008 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.802532 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:19.932201 21846 raft_consensus.cc:1261] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Refusing update from remote peer 157390243b4541548352b9d27da66cb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.932713 24482 consensus_queue.cc:1035] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:19.979806 21846 raft_consensus.cc:1261] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Refusing update from remote peer 9e6dd740161e46459fdb27c45ae2498e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.980248 24539 consensus_queue.cc:1035] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:19.995944 21710 raft_consensus.cc:1261] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer 157390243b4541548352b9d27da66cb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:19.996309 24481 consensus_queue.cc:1035] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:20.078472 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.079552 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.081786 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.083421 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.086508 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.094475 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.101157 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.139624 23051 consensus_queue.cc:237] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:20.140254 24619 raft_consensus.cc:2932] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:20.142820 21561 catalog_manager.cc:5484] T 1a3ca0726e154d74aa3da3afc065c827 P 90c9a86c10804941bfe2590985250c75 reported cstate change: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New cstate: current_term: 2 leader_uuid: "90c9a86c10804941bfe2590985250c75" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:20.145294 21823 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a3ca0726e154d74aa3da3afc065c827 with delete_type TABLET_DATA_TOMBSTONED (TS c5e97283120f47719b7fcec14be91dfe not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55542
I20240502 01:58:20.146116 24719 tablet_replica.cc:330] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: stopping tablet replica
I20240502 01:58:20.146438 24719 raft_consensus.cc:2229] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.146612 24719 raft_consensus.cc:2258] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.147819 24719 ts_tablet_manager.cc:1875] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.149777 24719 ts_tablet_manager.cc:1888] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.149858 24719 log.cc:1196] T 1a3ca0726e154d74aa3da3afc065c827 P c5e97283120f47719b7fcec14be91dfe: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/1a3ca0726e154d74aa3da3afc065c827
I20240502 01:58:20.150413 21546 catalog_manager.cc:4830] TS c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977): tablet 1a3ca0726e154d74aa3da3afc065c827 (table rebalance_test_table_1 [id=39421104a2574a2ab88931f1d0b154e9]) successfully deleted
I20240502 01:58:20.197372 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.198808 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.201393 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.202962 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.208256 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.209281 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.230206 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.266252 22916 consensus_queue.cc:237] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:20.266642 21710 raft_consensus.cc:1261] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer 90c9a86c10804941bfe2590985250c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:20.267056 24619 consensus_queue.cc:1035] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:20.267082 24107 raft_consensus.cc:2932] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:20.269198 21561 catalog_manager.cc:5484] T f2f2a1efb8da4f32ad8a2e74610892dd P f32583d958864898bc7e3567aed4334e reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "f32583d958864898bc7e3567aed4334e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:20.271811 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f2a1efb8da4f32ad8a2e74610892dd with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:20.272221 24782 tablet_replica.cc:330] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:20.272567 24782 raft_consensus.cc:2229] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.272732 24782 raft_consensus.cc:2258] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.273789 24782 ts_tablet_manager.cc:1875] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.275748 24782 ts_tablet_manager.cc:1888] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.275843 24782 log.cc:1196] T f2f2a1efb8da4f32ad8a2e74610892dd P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/f2f2a1efb8da4f32ad8a2e74610892dd
I20240502 01:58:20.276376 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet f2f2a1efb8da4f32ad8a2e74610892dd (table rebalance_test_table_2 [id=e32c50be8d624374b017ee7e8ced8664]) successfully deleted
I20240502 01:58:20.323539 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.325145 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.325881 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.327710 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.330055 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.332387 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.354081 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.385208 23306 consensus_queue.cc:237] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:20.385927 24566 raft_consensus.cc:2932] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:20.388339 21553 catalog_manager.cc:5484] T c03ce899e48c4447b56b1353c3f1d818 P d74b9a09ea17435298a8a6f2b82281e0 reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "d74b9a09ea17435298a8a6f2b82281e0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:20.390581 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet c03ce899e48c4447b56b1353c3f1d818 with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:20.390777 24782 tablet_replica.cc:330] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:20.390897 24782 raft_consensus.cc:2229] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.390996 24782 raft_consensus.cc:2258] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.391480 24782 ts_tablet_manager.cc:1875] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.393025 24782 ts_tablet_manager.cc:1888] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.393091 24782 log.cc:1196] T c03ce899e48c4447b56b1353c3f1d818 P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/c03ce899e48c4447b56b1353c3f1d818
I20240502 01:58:20.393496 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet c03ce899e48c4447b56b1353c3f1d818 (table rebalance_test_table_2 [id=e32c50be8d624374b017ee7e8ced8664]) successfully deleted
I20240502 01:58:20.442809 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.443975 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.444245 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.454387 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.456966 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.458616 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.462731 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.509254 23306 consensus_queue.cc:237] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } }
I20240502 01:58:20.509814 24836 raft_consensus.cc:2932] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:20.511947 21561 catalog_manager.cc:5484] T 8db3067333544aaf92267389358a6fbd P d74b9a09ea17435298a8a6f2b82281e0 reported cstate change: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New cstate: current_term: 2 leader_uuid: "d74b9a09ea17435298a8a6f2b82281e0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d74b9a09ea17435298a8a6f2b82281e0" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 35551 } } }
I20240502 01:58:20.514370 21823 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db3067333544aaf92267389358a6fbd with delete_type TABLET_DATA_TOMBSTONED (TS c5e97283120f47719b7fcec14be91dfe not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55542
I20240502 01:58:20.514612 24719 tablet_replica.cc:330] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: stopping tablet replica
I20240502 01:58:20.514739 24719 raft_consensus.cc:2229] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.514849 24719 raft_consensus.cc:2258] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.515542 24719 ts_tablet_manager.cc:1875] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.517503 24719 ts_tablet_manager.cc:1888] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.517583 24719 log.cc:1196] T 8db3067333544aaf92267389358a6fbd P c5e97283120f47719b7fcec14be91dfe: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/8db3067333544aaf92267389358a6fbd
I20240502 01:58:20.518013 21546 catalog_manager.cc:4830] TS c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977): tablet 8db3067333544aaf92267389358a6fbd (table rebalance_test_table_3 [id=a7aaed55cbdd49a89a6c397d097bb628]) successfully deleted
I20240502 01:58:20.565312 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.568850 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.570587 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.574734 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.583393 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.585444 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.600059 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.633728 23186 consensus_queue.cc:237] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:20.634366 24482 raft_consensus.cc:2932] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:20.636597 21552 catalog_manager.cc:5484] T 0ac53985ac124e23ac10c86b2b048aa8 P 157390243b4541548352b9d27da66cb7 reported cstate change: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New cstate: current_term: 2 leader_uuid: "157390243b4541548352b9d27da66cb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:20.639055 21823 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac53985ac124e23ac10c86b2b048aa8 with delete_type TABLET_DATA_TOMBSTONED (TS c5e97283120f47719b7fcec14be91dfe not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55542
I20240502 01:58:20.639261 24719 tablet_replica.cc:330] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: stopping tablet replica
I20240502 01:58:20.639384 24719 raft_consensus.cc:2229] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.639488 24719 raft_consensus.cc:2258] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.640087 24719 ts_tablet_manager.cc:1875] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.641944 24719 ts_tablet_manager.cc:1888] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.642009 24719 log.cc:1196] T 0ac53985ac124e23ac10c86b2b048aa8 P c5e97283120f47719b7fcec14be91dfe: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/0ac53985ac124e23ac10c86b2b048aa8
I20240502 01:58:20.642500 21546 catalog_manager.cc:4830] TS c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977): tablet 0ac53985ac124e23ac10c86b2b048aa8 (table rebalance_test_table_1 [id=39421104a2574a2ab88931f1d0b154e9]) successfully deleted
I20240502 01:58:20.688776 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.691874 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.693318 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.696656 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.702227 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.703258 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.708757 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.752055 22916 consensus_queue.cc:237] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } }
I20240502 01:58:20.752776 24776 raft_consensus.cc:2932] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:20.754914 21555 catalog_manager.cc:5484] T 8580c51fd90541c8bf70a2cee9c9ce36 P f32583d958864898bc7e3567aed4334e reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "f32583d958864898bc7e3567aed4334e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f32583d958864898bc7e3567aed4334e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 42143 } } }
I20240502 01:58:20.757457 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet 8580c51fd90541c8bf70a2cee9c9ce36 with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:20.757719 24782 tablet_replica.cc:330] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:20.757863 24782 raft_consensus.cc:2229] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.757977 24782 raft_consensus.cc:2258] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.758503 24782 ts_tablet_manager.cc:1875] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.760635 24782 ts_tablet_manager.cc:1888] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.760708 24782 log.cc:1196] T 8580c51fd90541c8bf70a2cee9c9ce36 P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/8580c51fd90541c8bf70a2cee9c9ce36
I20240502 01:58:20.761169 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet 8580c51fd90541c8bf70a2cee9c9ce36 (table rebalance_test_table_4 [id=e370f340e5c14b8a8efd4f0eade2363f]) successfully deleted
I20240502 01:58:20.810307 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.812316 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.812479 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.812834 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.813041 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.813524 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.822986 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.868918 23459 consensus_queue.cc:237] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:20.869542 24890 raft_consensus.cc:2932] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:20.871824 21553 catalog_manager.cc:5484] T f37d05f42cca45018b116b1729755c52 P 9e6dd740161e46459fdb27c45ae2498e reported cstate change: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New cstate: current_term: 2 leader_uuid: "9e6dd740161e46459fdb27c45ae2498e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:20.874404 21823 tablet_service.cc:1514] Processing DeleteTablet for tablet f37d05f42cca45018b116b1729755c52 with delete_type TABLET_DATA_TOMBSTONED (TS c5e97283120f47719b7fcec14be91dfe not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55542
I20240502 01:58:20.874644 24719 tablet_replica.cc:330] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: stopping tablet replica
I20240502 01:58:20.874778 24719 raft_consensus.cc:2229] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.874879 24719 raft_consensus.cc:2258] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.875458 24719 ts_tablet_manager.cc:1875] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.877234 24719 ts_tablet_manager.cc:1888] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.877300 24719 log.cc:1196] T f37d05f42cca45018b116b1729755c52 P c5e97283120f47719b7fcec14be91dfe: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/f37d05f42cca45018b116b1729755c52
I20240502 01:58:20.877756 21546 catalog_manager.cc:4830] TS c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977): tablet f37d05f42cca45018b116b1729755c52 (table rebalance_test_table_3 [id=a7aaed55cbdd49a89a6c397d097bb628]) successfully deleted
I20240502 01:58:20.922876 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.924384 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:20.925426 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:20.926712 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:20.940719 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.950701 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.951938 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:20.987319 23186 consensus_queue.cc:237] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } }
I20240502 01:58:20.987884 24482 raft_consensus.cc:2932] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:20.989717 21554 catalog_manager.cc:5484] T f0b94526b4414b758214e8d3800d2b53 P 157390243b4541548352b9d27da66cb7 reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "157390243b4541548352b9d27da66cb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "157390243b4541548352b9d27da66cb7" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 45485 } } }
I20240502 01:58:20.992132 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet f0b94526b4414b758214e8d3800d2b53 with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:20.992357 24782 tablet_replica.cc:330] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:20.992503 24782 raft_consensus.cc:2229] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:20.992619 24782 raft_consensus.cc:2258] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:20.993196 24782 ts_tablet_manager.cc:1875] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:20.995002 24782 ts_tablet_manager.cc:1888] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:20.995072 24782 log.cc:1196] T f0b94526b4414b758214e8d3800d2b53 P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/f0b94526b4414b758214e8d3800d2b53
I20240502 01:58:20.995529 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet f0b94526b4414b758214e8d3800d2b53 (table rebalance_test_table_0 [id=6fcab502c66b4b9cabfc4baaaa7dad37]) successfully deleted
I20240502 01:58:21.043100 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:21.043782 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:21.048943 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:21.056252 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:21.056782 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:21.062287 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:21.064931 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:21.104554 23051 consensus_queue.cc:237] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } }
I20240502 01:58:21.105113 24620 raft_consensus.cc:2932] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:21.107141 21554 catalog_manager.cc:5484] T b420b13c524544cd8560a93a7d5b6ee1 P 90c9a86c10804941bfe2590985250c75 reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "90c9a86c10804941bfe2590985250c75" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "90c9a86c10804941bfe2590985250c75" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 42373 } } }
I20240502 01:58:21.109484 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet b420b13c524544cd8560a93a7d5b6ee1 with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:21.109704 24782 tablet_replica.cc:330] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:21.109819 24782 raft_consensus.cc:2229] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:21.109902 24782 raft_consensus.cc:2258] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:21.110486 24782 ts_tablet_manager.cc:1875] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:21.112059 24782 ts_tablet_manager.cc:1888] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:21.112121 24782 log.cc:1196] T b420b13c524544cd8560a93a7d5b6ee1 P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/b420b13c524544cd8560a93a7d5b6ee1
I20240502 01:58:21.112542 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet b420b13c524544cd8560a93a7d5b6ee1 (table rebalance_test_table_0 [id=6fcab502c66b4b9cabfc4baaaa7dad37]) successfully deleted
I20240502 01:58:21.365316 21710 consensus_queue.cc:237] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
W20240502 01:58:21.368911 21642 consensus_peers.cc:485] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd -> Peer 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199:44415): Couldn't send request to peer 9e6dd740161e46459fdb27c45ae2498e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 630104b6bc4c49428f5a49561e396751. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:21.991518 25227 ts_tablet_manager.cc:902] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:21.991979 25227 tablet_copy_client.cc:308] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:21.995397 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet 630104b6bc4c49428f5a49561e396751 from peer 9e6dd740161e46459fdb27c45ae2498e ({username='slave'} at 127.19.43.199:50705)
I20240502 01:58:21.995499 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet 630104b6bc4c49428f5a49561e396751 from peer 9e6dd740161e46459fdb27c45ae2498e at {username='slave'} at 127.19.43.199:50705: session id = 9e6dd740161e46459fdb27c45ae2498e-630104b6bc4c49428f5a49561e396751
I20240502 01:58:21.996203 21731 tablet_copy_source_session.cc:205] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:21.996980 25227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630104b6bc4c49428f5a49561e396751. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:21.998759 25227 tablet_copy_client.cc:783] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:21.998898 25227 tablet_copy_client.cc:647] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:21.999905 25227 tablet_copy_client.cc:523] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:22.000873 25227 tablet_bootstrap.cc:492] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: Bootstrap starting.
I20240502 01:58:22.017755 25227 tablet_bootstrap.cc:492] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:22.017979 25227 tablet_bootstrap.cc:492] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: Bootstrap complete.
I20240502 01:58:22.018152 25227 ts_tablet_manager.cc:1367] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: Time spent bootstrapping tablet: real 0.017s user 0.018s sys 0.000s
I20240502 01:58:22.018617 25227 raft_consensus.cc:348] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:22.018808 25227 raft_consensus.cc:729] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e6dd740161e46459fdb27c45ae2498e, State: Initialized, Role: FOLLOWER
I20240502 01:58:22.018896 25227 consensus_queue.cc:260] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:22.019207 25227 ts_tablet_manager.cc:1398] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:22.019677 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session 9e6dd740161e46459fdb27c45ae2498e-630104b6bc4c49428f5a49561e396751 received from {username='slave'} at 127.19.43.199:50705
I20240502 01:58:22.019743 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session 9e6dd740161e46459fdb27c45ae2498e-630104b6bc4c49428f5a49561e396751 on tablet 630104b6bc4c49428f5a49561e396751 with peer 9e6dd740161e46459fdb27c45ae2498e
I20240502 01:58:22.581280 23459 raft_consensus.cc:1203] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:22.582207 25239 raft_consensus.cc:2932] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:22.582674 23459 raft_consensus.cc:2932] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:22.583724 25239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33898 (request call id 0) took 1218 ms. Trace:
I20240502 01:58:22.583796 25239 rpcz_store.cc:276] 0502 01:58:21.365001 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 01:58:21.365052 (+ 51us) service_pool.cc:225] Handling call
0502 01:58:21.365723 (+ 671us) log.cc:842] Serialized 260 byte log entry
0502 01:58:22.583703 (+1217980us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":38,"raft.queue_time_us":99,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":133,"spinlock_wait_cycles":3328,"thread_start_us":94,"threads_started":2,"wal-append.queue_time_us":125,"wal-append.run_cpu_time_us":311,"wal-append.run_wall_time_us":1000370}
I20240502 01:58:22.585553 21554 catalog_manager.cc:5484] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e reported cstate change: config changed from index -1 to 4, VOTER 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:22.638430 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:22.638876 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.639292 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:22.641165 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:22.644441 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.658304 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.669862 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.694439 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "630104b6bc4c49428f5a49561e396751"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:33912
I20240502 01:58:22.694588 21710 raft_consensus.cc:595] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:22.741386 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.741631 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.747627 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.752411 23439 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:22.754400 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:22.754402 21689 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:22.774688 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:22.982203 25237 raft_consensus.cc:979] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower 9e6dd740161e46459fdb27c45ae2498e to start an election
I20240502 01:58:22.982304 25237 raft_consensus.cc:1067] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer 9e6dd740161e46459fdb27c45ae2498e to start an election
I20240502 01:58:22.982749 23459 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "630104b6bc4c49428f5a49561e396751"
dest_uuid: "9e6dd740161e46459fdb27c45ae2498e"
from {username='slave'} at 127.19.43.193:58151
I20240502 01:58:22.982889 23459 raft_consensus.cc:482] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:22.982952 23459 raft_consensus.cc:3037] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:22.983737 23459 raft_consensus.cc:504] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:22.984102 23459 leader_election.cc:288] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:22.984440 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630104b6bc4c49428f5a49561e396751" candidate_uuid: "9e6dd740161e46459fdb27c45ae2498e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:22.984551 21710 raft_consensus.cc:3032] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:22.984587 21710 raft_consensus.cc:729] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:22.984707 21710 consensus_queue.cc:260] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:22.984843 21710 raft_consensus.cc:3037] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:22.985574 21710 raft_consensus.cc:2454] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e6dd740161e46459fdb27c45ae2498e in term 2.
I20240502 01:58:22.985888 23392 leader_election.cc:302] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd, 9e6dd740161e46459fdb27c45ae2498e; no voters:
I20240502 01:58:22.986105 25238 raft_consensus.cc:2781] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:22.986265 25238 raft_consensus.cc:686] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 2 LEADER]: Becoming Leader. State: Replica: 9e6dd740161e46459fdb27c45ae2498e, State: Running, Role: LEADER
I20240502 01:58:22.986418 25238 consensus_queue.cc:237] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:22.987897 21561 catalog_manager.cc:5484] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to 9e6dd740161e46459fdb27c45ae2498e (127.19.43.199). New cstate: current_term: 2 leader_uuid: "9e6dd740161e46459fdb27c45ae2498e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:23.044907 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.044981 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.051578 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.052649 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.053718 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.059051 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.061532 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.339962 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.343803 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.346550 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.350198 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.351965 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.352984 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.360770 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.373271 21710 raft_consensus.cc:1261] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer 9e6dd740161e46459fdb27c45ae2498e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:23.373663 25238 consensus_queue.cc:1035] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:23.639217 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.640683 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.641158 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.641690 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.662640 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.663158 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.663784 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.703465 23459 consensus_queue.cc:237] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } }
I20240502 01:58:23.704016 25335 raft_consensus.cc:2932] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:23.705955 21559 catalog_manager.cc:5484] T 630104b6bc4c49428f5a49561e396751 P 9e6dd740161e46459fdb27c45ae2498e reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "9e6dd740161e46459fdb27c45ae2498e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e6dd740161e46459fdb27c45ae2498e" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44415 } } }
I20240502 01:58:23.708412 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet 630104b6bc4c49428f5a49561e396751 with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:23.708719 25473 tablet_replica.cc:330] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:23.708863 25473 raft_consensus.cc:2229] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:23.708964 25473 raft_consensus.cc:2258] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:23.709506 25473 ts_tablet_manager.cc:1875] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:23.711144 25473 ts_tablet_manager.cc:1888] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:23.711208 25473 log.cc:1196] T 630104b6bc4c49428f5a49561e396751 P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/630104b6bc4c49428f5a49561e396751
I20240502 01:58:23.711635 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet 630104b6bc4c49428f5a49561e396751 (table rebalance_test_table_4 [id=e370f340e5c14b8a8efd4f0eade2363f]) successfully deleted
I20240502 01:58:23.966811 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.967936 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.971123 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:23.977784 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.979319 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:23.980490 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.000424 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.067974 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.069598 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.074311 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.074314 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.082289 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.082937 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.082940 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.295877 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.298166 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.300683 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.301283 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.301544 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.303072 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.321936 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.415953 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.416822 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.418368 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.421473 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.422116 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.430208 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.442410 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.529480 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.529887 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:24.532450 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.534164 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.548519 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.549466 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:24.549489 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
c1be09c507b24ad78d60f64e8e64d7a6 | 127.19.43.254:42289 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.19.43.212:42513 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
157390243b4541548352b9d27da66cb7 | 127.19.43.197:45485 | HEALTHY | <none> | 2 | 0
1ab91a4d0007452a85f25102557ed3dd | 127.19.43.193:42561 | HEALTHY | <none> | 3 | 0
90c9a86c10804941bfe2590985250c75 | 127.19.43.196:42373 | HEALTHY | <none> | 2 | 0
9e6dd740161e46459fdb27c45ae2498e | 127.19.43.199:44415 | HEALTHY | <none> | 2 | 0
c5e97283120f47719b7fcec14be91dfe | 127.19.43.194:43977 | HEALTHY | <none> | 3 | 0
d74b9a09ea17435298a8a6f2b82281e0 | 127.19.43.198:35551 | HEALTHY | <none> | 2 | 0
f32583d958864898bc7e3567aed4334e | 127.19.43.195:42143 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.19.43.193 | experimental | 127.19.43.193:42561
local_ip_for_outbound_sockets | 127.19.43.194 | experimental | 127.19.43.194:43977
local_ip_for_outbound_sockets | 127.19.43.195 | experimental | 127.19.43.195:42143
local_ip_for_outbound_sockets | 127.19.43.196 | experimental | 127.19.43.196:42373
local_ip_for_outbound_sockets | 127.19.43.197 | experimental | 127.19.43.197:45485
local_ip_for_outbound_sockets | 127.19.43.198 | experimental | 127.19.43.198:35551
local_ip_for_outbound_sockets | 127.19.43.199 | experimental | 127.19.43.199:44415
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.19.43.193:42561
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.19.43.194:43977
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.19.43.195:42143
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.19.43.196:42373
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.19.43.197:45485
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.19.43.198:35551
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.19.43.199:44415
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.19.43.212:42513 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 01:58:24.581185 19631 log_verifier.cc:126] Checking tablet 0ac53985ac124e23ac10c86b2b048aa8
I20240502 01:58:24.591666 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 0ac53985ac124e23ac10c86b2b048aa8
I20240502 01:58:24.591737 19631 log_verifier.cc:126] Checking tablet 1a3ca0726e154d74aa3da3afc065c827
I20240502 01:58:24.599516 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 1a3ca0726e154d74aa3da3afc065c827
I20240502 01:58:24.599558 19631 log_verifier.cc:126] Checking tablet 39b14aa4387a4d30854f4886383a9452
I20240502 01:58:24.607357 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 39b14aa4387a4d30854f4886383a9452
I20240502 01:58:24.607398 19631 log_verifier.cc:126] Checking tablet 3bd57c7e02a74d5988cf68d7546ca770
I20240502 01:58:24.615415 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 3bd57c7e02a74d5988cf68d7546ca770
I20240502 01:58:24.615458 19631 log_verifier.cc:126] Checking tablet 4fef85fe1e5c4dc1bb810d3c93742756
I20240502 01:58:24.623291 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4fef85fe1e5c4dc1bb810d3c93742756
I20240502 01:58:24.623342 19631 log_verifier.cc:126] Checking tablet 5d99be4a87b24089bc4185904dc8fd30
I20240502 01:58:24.631444 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 5d99be4a87b24089bc4185904dc8fd30
I20240502 01:58:24.631485 19631 log_verifier.cc:126] Checking tablet 630104b6bc4c49428f5a49561e396751
I20240502 01:58:24.639519 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 630104b6bc4c49428f5a49561e396751
I20240502 01:58:24.639564 19631 log_verifier.cc:126] Checking tablet 6935b7c50570412691bb3086b2b7a370
I20240502 01:58:24.647584 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 6935b7c50570412691bb3086b2b7a370
I20240502 01:58:24.647624 19631 log_verifier.cc:126] Checking tablet 8580c51fd90541c8bf70a2cee9c9ce36
I20240502 01:58:24.655712 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8580c51fd90541c8bf70a2cee9c9ce36
I20240502 01:58:24.655755 19631 log_verifier.cc:126] Checking tablet 8db3067333544aaf92267389358a6fbd
I20240502 01:58:24.664067 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8db3067333544aaf92267389358a6fbd
I20240502 01:58:24.664110 19631 log_verifier.cc:126] Checking tablet 97be852ca8c241c1a44c3e4a66490ca9
I20240502 01:58:24.671350 19631 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 97be852ca8c241c1a44c3e4a66490ca9
I20240502 01:58:24.671392 19631 log_verifier.cc:126] Checking tablet b420b13c524544cd8560a93a7d5b6ee1
I20240502 01:58:24.679512 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet b420b13c524544cd8560a93a7d5b6ee1
I20240502 01:58:24.679556 19631 log_verifier.cc:126] Checking tablet c03ce899e48c4447b56b1353c3f1d818
I20240502 01:58:24.687867 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet c03ce899e48c4447b56b1353c3f1d818
I20240502 01:58:24.687924 19631 log_verifier.cc:126] Checking tablet f0b94526b4414b758214e8d3800d2b53
I20240502 01:58:24.696056 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f0b94526b4414b758214e8d3800d2b53
I20240502 01:58:24.696100 19631 log_verifier.cc:126] Checking tablet f2f2a1efb8da4f32ad8a2e74610892dd
I20240502 01:58:24.703951 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f2f2a1efb8da4f32ad8a2e74610892dd
I20240502 01:58:24.703994 19631 log_verifier.cc:126] Checking tablet f37d05f42cca45018b116b1729755c52
I20240502 01:58:24.711880 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f37d05f42cca45018b116b1729755c52
I20240502 01:58:24.713526 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.200:0
--local_ip_for_outbound_sockets=127.19.43.200
--webserver_interface=127.19.43.200
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--builtin_ntp_servers=127.19.43.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:24.813818 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:24.813993 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:24.814074 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:24.816967 25652 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:24.817023 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:24.817101 25652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.200
I20240502 01:58:24.820545 25652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.200:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.19.43.200
--webserver_port=0
--tserver_master_addrs=127.19.43.254:42289
--never_fsync=true
--heap_profile_path=/tmp/kudu.25652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.200
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:24.821273 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:24.822299 25652 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:24.829185 25659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:24.829164 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:24.829290 25652 server_base.cc:1034] running on GCE node
W20240502 01:58:24.829169 25660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:24.829782 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:24.830294 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:24.831440 25652 hybrid_clock.cc:648] HybridClock initialized: now 1714615104831422 us; error 31 us; skew 500 ppm
I20240502 01:58:24.832958 25652 webserver.cc:460] Webserver started at http://127.19.43.200:34301/ using document root <none> and password file <none>
I20240502 01:58:24.833442 25652 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:24.833492 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:24.833653 25652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:24.835122 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "24990c20b5214d509ae74b9a90178ea1"
format_stamp: "Formatted at 2024-05-02 01:58:24 on dist-test-slave-mnd3"
I20240502 01:58:24.835544 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "24990c20b5214d509ae74b9a90178ea1"
format_stamp: "Formatted at 2024-05-02 01:58:24 on dist-test-slave-mnd3"
I20240502 01:58:24.838757 25652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 01:58:24.840725 25668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:24.841738 25652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:24.841835 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "24990c20b5214d509ae74b9a90178ea1"
format_stamp: "Formatted at 2024-05-02 01:58:24 on dist-test-slave-mnd3"
I20240502 01:58:24.841909 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:24.858217 25652 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:24.858695 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:24.858803 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:24.859246 25652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:24.860082 25652 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:24.860136 25652 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:24.860173 25652 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:24.860193 25652 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:24.867921 25652 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.200:45989
I20240502 01:58:24.867969 25787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.200:45989 every 8 connection(s)
I20240502 01:58:24.868681 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 01:58:24.875874 25788 heartbeater.cc:347] Connected to a master server at 127.19.43.254:42289
I20240502 01:58:24.876053 25788 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:24.876413 25788 heartbeater.cc:511] Master 127.19.43.254:42289 requested a full tablet report, sending...
I20240502 01:58:24.877230 21559 ts_manager.cc:194] Registered new tserver with Master: 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200:45989)
I20240502 01:58:24.877765 21559 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.200:45801
I20240502 01:58:24.878159 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 25652
I20240502 01:58:24.878260 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 01:58:25.052424 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.053264 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.054517 25720 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:25.061875 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:25.062327 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.063758 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:25.073257 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.083099 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.185062 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.185163 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.195712 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:25.196298 25720 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:25.197228 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.197896 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:25.204246 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.222129 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:25.258455 21846 consensus_queue.cc:237] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
W20240502 01:58:25.263032 21777 consensus_peers.cc:485] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe -> Peer 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200:45989): Couldn't send request to peer 24990c20b5214d509ae74b9a90178ea1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fef85fe1e5c4dc1bb810d3c93742756. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:25.724336 25895 ts_tablet_manager.cc:902] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Initiating tablet copy from peer c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:25.724963 25895 tablet_copy_client.cc:308] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:43977
I20240502 01:58:25.728499 21866 tablet_copy_service.cc:140] P c5e97283120f47719b7fcec14be91dfe: Received BeginTabletCopySession request for tablet 4fef85fe1e5c4dc1bb810d3c93742756 from peer 24990c20b5214d509ae74b9a90178ea1 ({username='slave'} at 127.19.43.200:40415)
I20240502 01:58:25.728600 21866 tablet_copy_service.cc:161] P c5e97283120f47719b7fcec14be91dfe: Beginning new tablet copy session on tablet 4fef85fe1e5c4dc1bb810d3c93742756 from peer 24990c20b5214d509ae74b9a90178ea1 at {username='slave'} at 127.19.43.200:40415: session id = 24990c20b5214d509ae74b9a90178ea1-4fef85fe1e5c4dc1bb810d3c93742756
I20240502 01:58:25.729282 21866 tablet_copy_source_session.cc:205] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:25.732077 25895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef85fe1e5c4dc1bb810d3c93742756. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:25.736552 25895 tablet_copy_client.cc:783] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:25.736708 25895 tablet_copy_client.cc:647] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:25.737993 25895 tablet_copy_client.cc:523] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:25.739346 25895 tablet_bootstrap.cc:492] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Bootstrap starting.
I20240502 01:58:25.744076 25895 log.cc:824] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:25.762430 25895 tablet_bootstrap.cc:492] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:25.762704 25895 tablet_bootstrap.cc:492] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Bootstrap complete.
I20240502 01:58:25.762938 25895 ts_tablet_manager.cc:1367] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Time spent bootstrapping tablet: real 0.024s user 0.024s sys 0.000s
I20240502 01:58:25.764549 25895 raft_consensus.cc:348] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } }
I20240502 01:58:25.765019 25895 raft_consensus.cc:729] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24990c20b5214d509ae74b9a90178ea1, State: Initialized, Role: FOLLOWER
I20240502 01:58:25.765272 25895 consensus_queue.cc:260] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:25.765760 25788 heartbeater.cc:503] Master 127.19.43.254:42289 was elected leader, sending a full tablet report...
I20240502 01:58:25.765949 25895 ts_tablet_manager.cc:1398] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:25.766510 21866 tablet_copy_service.cc:342] P c5e97283120f47719b7fcec14be91dfe: Request end of tablet copy session 24990c20b5214d509ae74b9a90178ea1-4fef85fe1e5c4dc1bb810d3c93742756 received from {username='slave'} at 127.19.43.200:40415
I20240502 01:58:25.766579 21866 tablet_copy_service.cc:433] P c5e97283120f47719b7fcec14be91dfe: ending tablet copy session 24990c20b5214d509ae74b9a90178ea1-4fef85fe1e5c4dc1bb810d3c93742756 on tablet 4fef85fe1e5c4dc1bb810d3c93742756 with peer 24990c20b5214d509ae74b9a90178ea1
I20240502 01:58:26.253185 25741 raft_consensus.cc:1203] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:26.254319 25904 raft_consensus.cc:2932] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:26.255494 25741 raft_consensus.cc:2932] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:26.256542 21553 catalog_manager.cc:5484] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe reported cstate change: config changed from index -1 to 4, VOTER 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "c5e97283120f47719b7fcec14be91dfe" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:26.277727 21710 consensus_queue.cc:237] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
W20240502 01:58:26.281443 21640 consensus_peers.cc:485] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd -> Peer 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200:45989): Couldn't send request to peer 24990c20b5214d509ae74b9a90178ea1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6935b7c50570412691bb3086b2b7a370. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:26.814549 25933 ts_tablet_manager.cc:902] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: Initiating tablet copy from peer 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:26.815021 25933 tablet_copy_client.cc:308] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:42561
I20240502 01:58:26.818502 21731 tablet_copy_service.cc:140] P 1ab91a4d0007452a85f25102557ed3dd: Received BeginTabletCopySession request for tablet 6935b7c50570412691bb3086b2b7a370 from peer 24990c20b5214d509ae74b9a90178ea1 ({username='slave'} at 127.19.43.200:43143)
I20240502 01:58:26.818598 21731 tablet_copy_service.cc:161] P 1ab91a4d0007452a85f25102557ed3dd: Beginning new tablet copy session on tablet 6935b7c50570412691bb3086b2b7a370 from peer 24990c20b5214d509ae74b9a90178ea1 at {username='slave'} at 127.19.43.200:43143: session id = 24990c20b5214d509ae74b9a90178ea1-6935b7c50570412691bb3086b2b7a370
I20240502 01:58:26.819322 21731 tablet_copy_source_session.cc:205] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:26.820124 25933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6935b7c50570412691bb3086b2b7a370. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:26.821911 25933 tablet_copy_client.cc:783] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:26.822083 25933 tablet_copy_client.cc:647] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:26.823137 25933 tablet_copy_client.cc:523] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:26.824092 25933 tablet_bootstrap.cc:492] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: Bootstrap starting.
I20240502 01:58:26.840831 25933 tablet_bootstrap.cc:492] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:58:26.841028 25933 tablet_bootstrap.cc:492] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: Bootstrap complete.
I20240502 01:58:26.841146 25933 ts_tablet_manager.cc:1367] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20240502 01:58:26.841557 25933 raft_consensus.cc:348] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } }
I20240502 01:58:26.841742 25933 raft_consensus.cc:729] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24990c20b5214d509ae74b9a90178ea1, State: Initialized, Role: FOLLOWER
I20240502 01:58:26.841837 25933 consensus_queue.cc:260] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:26.842156 25933 ts_tablet_manager.cc:1398] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1: Time spent starting tablet: real 0.001s user 0.002s sys 0.002s
I20240502 01:58:26.842607 21731 tablet_copy_service.cc:342] P 1ab91a4d0007452a85f25102557ed3dd: Request end of tablet copy session 24990c20b5214d509ae74b9a90178ea1-6935b7c50570412691bb3086b2b7a370 received from {username='slave'} at 127.19.43.200:43143
I20240502 01:58:26.842680 21731 tablet_copy_service.cc:433] P 1ab91a4d0007452a85f25102557ed3dd: ending tablet copy session 24990c20b5214d509ae74b9a90178ea1-6935b7c50570412691bb3086b2b7a370 on tablet 6935b7c50570412691bb3086b2b7a370 with peer 24990c20b5214d509ae74b9a90178ea1
I20240502 01:58:27.243019 25741 raft_consensus.cc:1203] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:27.243856 25942 raft_consensus.cc:2932] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:27.244413 25741 raft_consensus.cc:2932] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:27.246614 21552 catalog_manager.cc:5484] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd reported cstate change: config changed from index -1 to 4, VOTER 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "1ab91a4d0007452a85f25102557ed3dd" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:27.300257 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.302578 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.303762 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.307749 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.311698 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.313782 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.336436 25720 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:27.342097 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.361256 21710 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6935b7c50570412691bb3086b2b7a370"
dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:34036
I20240502 01:58:27.361382 21710 raft_consensus.cc:595] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:27.408608 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.408875 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.409366 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.416669 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.416802 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.419456 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.426322 25720 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:27.439026 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.514437 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.517124 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.518829 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.519914 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.523798 25720 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:27.535288 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.535629 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.536581 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.567674 21846 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4fef85fe1e5c4dc1bb810d3c93742756"
dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57150
I20240502 01:58:27.567807 21846 raft_consensus.cc:595] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:27.615027 25720 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:27.615593 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.618857 21823 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.619623 21689 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:27.619866 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.627813 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.634853 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.637135 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.756927 25976 raft_consensus.cc:979] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: : Instructing follower 24990c20b5214d509ae74b9a90178ea1 to start an election
I20240502 01:58:27.757025 25976 raft_consensus.cc:1067] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Signalling peer 24990c20b5214d509ae74b9a90178ea1 to start an election
I20240502 01:58:27.757463 25741 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4fef85fe1e5c4dc1bb810d3c93742756"
dest_uuid: "24990c20b5214d509ae74b9a90178ea1"
from {username='slave'} at 127.19.43.194:39761
I20240502 01:58:27.757643 25741 raft_consensus.cc:482] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:27.757725 25741 raft_consensus.cc:3037] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:27.758582 25741 raft_consensus.cc:504] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:27.759438 25741 leader_election.cc:288] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [CANDIDATE]: Term 2 election: Requested vote from peers c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977)
I20240502 01:58:27.759766 21846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef85fe1e5c4dc1bb810d3c93742756" candidate_uuid: "24990c20b5214d509ae74b9a90178ea1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c5e97283120f47719b7fcec14be91dfe"
I20240502 01:58:27.759876 21846 raft_consensus.cc:3032] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:27.759927 21846 raft_consensus.cc:729] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5e97283120f47719b7fcec14be91dfe, State: Running, Role: LEADER
I20240502 01:58:27.760044 21846 consensus_queue.cc:260] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:27.760164 21846 raft_consensus.cc:3037] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:27.760933 21846 raft_consensus.cc:2454] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24990c20b5214d509ae74b9a90178ea1 in term 2.
I20240502 01:58:27.761307 25670 leader_election.cc:302] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 24990c20b5214d509ae74b9a90178ea1, c5e97283120f47719b7fcec14be91dfe; no voters:
I20240502 01:58:27.761627 26120 raft_consensus.cc:2781] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:27.761787 26120 raft_consensus.cc:686] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 2 LEADER]: Becoming Leader. State: Replica: 24990c20b5214d509ae74b9a90178ea1, State: Running, Role: LEADER
I20240502 01:58:27.762094 26120 consensus_queue.cc:237] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:27.764295 21559 catalog_manager.cc:5484] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 reported cstate change: term changed from 1 to 2, leader changed from c5e97283120f47719b7fcec14be91dfe (127.19.43.194) to 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200). New cstate: current_term: 2 leader_uuid: "24990c20b5214d509ae74b9a90178ea1" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:27.837332 26126 raft_consensus.cc:979] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: : Instructing follower 24990c20b5214d509ae74b9a90178ea1 to start an election
I20240502 01:58:27.837430 26126 raft_consensus.cc:1067] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Signalling peer 24990c20b5214d509ae74b9a90178ea1 to start an election
I20240502 01:58:27.837846 25741 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6935b7c50570412691bb3086b2b7a370"
dest_uuid: "24990c20b5214d509ae74b9a90178ea1"
from {username='slave'} at 127.19.43.193:49137
I20240502 01:58:27.837973 25741 raft_consensus.cc:482] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:27.838043 25741 raft_consensus.cc:3037] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:27.838841 25741 raft_consensus.cc:504] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:27.839167 25741 leader_election.cc:288] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [CANDIDATE]: Term 2 election: Requested vote from peers 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561)
I20240502 01:58:27.839504 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6935b7c50570412691bb3086b2b7a370" candidate_uuid: "24990c20b5214d509ae74b9a90178ea1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ab91a4d0007452a85f25102557ed3dd"
I20240502 01:58:27.839617 21710 raft_consensus.cc:3032] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:27.839663 21710 raft_consensus.cc:729] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ab91a4d0007452a85f25102557ed3dd, State: Running, Role: LEADER
I20240502 01:58:27.839800 21710 consensus_queue.cc:260] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:27.839951 21710 raft_consensus.cc:3037] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:27.840662 21710 raft_consensus.cc:2454] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24990c20b5214d509ae74b9a90178ea1 in term 2.
I20240502 01:58:27.840966 25672 leader_election.cc:302] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1ab91a4d0007452a85f25102557ed3dd, 24990c20b5214d509ae74b9a90178ea1; no voters:
I20240502 01:58:27.841137 26120 raft_consensus.cc:2781] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:27.841220 26120 raft_consensus.cc:686] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 2 LEADER]: Becoming Leader. State: Replica: 24990c20b5214d509ae74b9a90178ea1, State: Running, Role: LEADER
I20240502 01:58:27.841341 26120 consensus_queue.cc:237] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:27.842639 21559 catalog_manager.cc:5484] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 reported cstate change: term changed from 1 to 2, leader changed from 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) to 24990c20b5214d509ae74b9a90178ea1 (127.19.43.200). New cstate: current_term: 2 leader_uuid: "24990c20b5214d509ae74b9a90178ea1" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 } } peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:27.922004 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.923645 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.931535 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.932477 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.933643 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.939190 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.951076 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:27.956250 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.030792 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.035066 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.035261 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.035915 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.038926 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.043957 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.048504 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.058936 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.260128 21846 raft_consensus.cc:1261] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Refusing update from remote peer 24990c20b5214d509ae74b9a90178ea1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:28.260676 26120 consensus_queue.cc:1035] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e97283120f47719b7fcec14be91dfe" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:28.333200 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.337316 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.337749 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.341415 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.345561 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.356415 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.365993 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.372249 21710 raft_consensus.cc:1261] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Refusing update from remote peer 24990c20b5214d509ae74b9a90178ea1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:58:28.372732 26121 consensus_queue.cc:1035] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab91a4d0007452a85f25102557ed3dd" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:28.377725 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.445963 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.446770 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.448841 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.449013 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.449307 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.463706 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.466560 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.476873 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.516018 25741 consensus_queue.cc:237] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:28.516570 26120 raft_consensus.cc:2932] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:28.518795 21559 catalog_manager.cc:5484] T 4fef85fe1e5c4dc1bb810d3c93742756 P 24990c20b5214d509ae74b9a90178ea1 reported cstate change: config changed from index 4 to 6, VOTER c5e97283120f47719b7fcec14be91dfe (127.19.43.194) evicted. New cstate: current_term: 2 leader_uuid: "24990c20b5214d509ae74b9a90178ea1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:28.521127 21823 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fef85fe1e5c4dc1bb810d3c93742756 with delete_type TABLET_DATA_TOMBSTONED (TS c5e97283120f47719b7fcec14be91dfe not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55542
I20240502 01:58:28.521489 26310 tablet_replica.cc:330] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: stopping tablet replica
I20240502 01:58:28.521610 26310 raft_consensus.cc:2229] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:28.521716 26310 raft_consensus.cc:2258] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:28.522876 26310 ts_tablet_manager.cc:1875] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:28.524531 26310 ts_tablet_manager.cc:1888] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:28.524597 26310 log.cc:1196] T 4fef85fe1e5c4dc1bb810d3c93742756 P c5e97283120f47719b7fcec14be91dfe: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/4fef85fe1e5c4dc1bb810d3c93742756
I20240502 01:58:28.525022 21546 catalog_manager.cc:4830] TS c5e97283120f47719b7fcec14be91dfe (127.19.43.194:43977): tablet 4fef85fe1e5c4dc1bb810d3c93742756 (table rebalance_test_table_1 [id=39421104a2574a2ab88931f1d0b154e9]) successfully deleted
I20240502 01:58:28.770727 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.771360 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.780720 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.780849 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.782166 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.782584 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.786289 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.812387 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.844615 25741 consensus_queue.cc:237] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } }
I20240502 01:58:28.845263 26120 raft_consensus.cc:2932] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:28.848675 21552 catalog_manager.cc:5484] T 6935b7c50570412691bb3086b2b7a370 P 24990c20b5214d509ae74b9a90178ea1 reported cstate change: config changed from index 4 to 6, VOTER 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "24990c20b5214d509ae74b9a90178ea1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "24990c20b5214d509ae74b9a90178ea1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 45989 } } }
I20240502 01:58:28.851070 21689 tablet_service.cc:1514] Processing DeleteTablet for tablet 6935b7c50570412691bb3086b2b7a370 with delete_type TABLET_DATA_TOMBSTONED (TS 1ab91a4d0007452a85f25102557ed3dd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55034
I20240502 01:58:28.851447 26366 tablet_replica.cc:330] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: stopping tablet replica
I20240502 01:58:28.851583 26366 raft_consensus.cc:2229] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:28.851755 26366 raft_consensus.cc:2258] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:28.853296 26366 ts_tablet_manager.cc:1875] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:28.855536 26366 ts_tablet_manager.cc:1888] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:58:28.855605 26366 log.cc:1196] T 6935b7c50570412691bb3086b2b7a370 P 1ab91a4d0007452a85f25102557ed3dd: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/6935b7c50570412691bb3086b2b7a370
I20240502 01:58:28.856145 21548 catalog_manager.cc:4830] TS 1ab91a4d0007452a85f25102557ed3dd (127.19.43.193:42561): tablet 6935b7c50570412691bb3086b2b7a370 (table rebalance_test_table_4 [id=e370f340e5c14b8a8efd4f0eade2363f]) successfully deleted
I20240502 01:58:28.918493 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.920534 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.920734 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.929175 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.934239 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.936231 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.948379 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:28.957435 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.028942 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.030120 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.036366 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.038434 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.039222 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.046326 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.059036 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.066243 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.151085 25720 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.151918 23166 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.152580 21689 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.155042 23030 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.155898 21823 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.158978 23439 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.168051 23301 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:29.191395 22894 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
c1be09c507b24ad78d60f64e8e64d7a6 | 127.19.43.254:42289 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.19.43.212:42513 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
157390243b4541548352b9d27da66cb7 | 127.19.43.197:45485 | HEALTHY | <none> | 2 | 0
1ab91a4d0007452a85f25102557ed3dd | 127.19.43.193:42561 | HEALTHY | <none> | 2 | 0
24990c20b5214d509ae74b9a90178ea1 | 127.19.43.200:45989 | HEALTHY | <none> | 2 | 0
90c9a86c10804941bfe2590985250c75 | 127.19.43.196:42373 | HEALTHY | <none> | 2 | 0
9e6dd740161e46459fdb27c45ae2498e | 127.19.43.199:44415 | HEALTHY | <none> | 2 | 0
c5e97283120f47719b7fcec14be91dfe | 127.19.43.194:43977 | HEALTHY | <none> | 2 | 0
d74b9a09ea17435298a8a6f2b82281e0 | 127.19.43.198:35551 | HEALTHY | <none> | 2 | 0
f32583d958864898bc7e3567aed4334e | 127.19.43.195:42143 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.19.43.193 | experimental | 127.19.43.193:42561
local_ip_for_outbound_sockets | 127.19.43.194 | experimental | 127.19.43.194:43977
local_ip_for_outbound_sockets | 127.19.43.195 | experimental | 127.19.43.195:42143
local_ip_for_outbound_sockets | 127.19.43.196 | experimental | 127.19.43.196:42373
local_ip_for_outbound_sockets | 127.19.43.197 | experimental | 127.19.43.197:45485
local_ip_for_outbound_sockets | 127.19.43.198 | experimental | 127.19.43.198:35551
local_ip_for_outbound_sockets | 127.19.43.199 | experimental | 127.19.43.199:44415
local_ip_for_outbound_sockets | 127.19.43.200 | experimental | 127.19.43.200:45989
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.19.43.193:42561
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.19.43.194:43977
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.19.43.195:42143
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.19.43.196:42373
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.19.43.197:45485
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.19.43.198:35551
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.19.43.199:44415
server_dump_info_path | /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.19.43.200:45989
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.19.43.212:42513 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 2
Maximum | 2
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 01:58:29.209513 19631 log_verifier.cc:126] Checking tablet 0ac53985ac124e23ac10c86b2b048aa8
I20240502 01:58:29.215740 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 0ac53985ac124e23ac10c86b2b048aa8
I20240502 01:58:29.215793 19631 log_verifier.cc:126] Checking tablet 1a3ca0726e154d74aa3da3afc065c827
I20240502 01:58:29.221309 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 1a3ca0726e154d74aa3da3afc065c827
I20240502 01:58:29.221354 19631 log_verifier.cc:126] Checking tablet 39b14aa4387a4d30854f4886383a9452
I20240502 01:58:29.226562 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 39b14aa4387a4d30854f4886383a9452
I20240502 01:58:29.226604 19631 log_verifier.cc:126] Checking tablet 3bd57c7e02a74d5988cf68d7546ca770
I20240502 01:58:29.231799 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 3bd57c7e02a74d5988cf68d7546ca770
I20240502 01:58:29.231842 19631 log_verifier.cc:126] Checking tablet 4fef85fe1e5c4dc1bb810d3c93742756
I20240502 01:58:29.239794 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 4fef85fe1e5c4dc1bb810d3c93742756
I20240502 01:58:29.239841 19631 log_verifier.cc:126] Checking tablet 5d99be4a87b24089bc4185904dc8fd30
I20240502 01:58:29.245548 19631 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 5d99be4a87b24089bc4185904dc8fd30
I20240502 01:58:29.245596 19631 log_verifier.cc:126] Checking tablet 630104b6bc4c49428f5a49561e396751
I20240502 01:58:29.252542 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 630104b6bc4c49428f5a49561e396751
I20240502 01:58:29.252593 19631 log_verifier.cc:126] Checking tablet 6935b7c50570412691bb3086b2b7a370
I20240502 01:58:29.262569 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 6935b7c50570412691bb3086b2b7a370
I20240502 01:58:29.262629 19631 log_verifier.cc:126] Checking tablet 8580c51fd90541c8bf70a2cee9c9ce36
I20240502 01:58:29.268301 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8580c51fd90541c8bf70a2cee9c9ce36
I20240502 01:58:29.268344 19631 log_verifier.cc:126] Checking tablet 8db3067333544aaf92267389358a6fbd
I20240502 01:58:29.273595 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8db3067333544aaf92267389358a6fbd
I20240502 01:58:29.273638 19631 log_verifier.cc:126] Checking tablet 97be852ca8c241c1a44c3e4a66490ca9
I20240502 01:58:29.277815 19631 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 97be852ca8c241c1a44c3e4a66490ca9
I20240502 01:58:29.277854 19631 log_verifier.cc:126] Checking tablet b420b13c524544cd8560a93a7d5b6ee1
I20240502 01:58:29.283345 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet b420b13c524544cd8560a93a7d5b6ee1
I20240502 01:58:29.283385 19631 log_verifier.cc:126] Checking tablet c03ce899e48c4447b56b1353c3f1d818
I20240502 01:58:29.288794 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet c03ce899e48c4447b56b1353c3f1d818
I20240502 01:58:29.288832 19631 log_verifier.cc:126] Checking tablet f0b94526b4414b758214e8d3800d2b53
I20240502 01:58:29.294166 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f0b94526b4414b758214e8d3800d2b53
I20240502 01:58:29.294217 19631 log_verifier.cc:126] Checking tablet f2f2a1efb8da4f32ad8a2e74610892dd
I20240502 01:58:29.299501 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f2f2a1efb8da4f32ad8a2e74610892dd
I20240502 01:58:29.299540 19631 log_verifier.cc:126] Checking tablet f37d05f42cca45018b116b1729755c52
I20240502 01:58:29.304813 19631 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f37d05f42cca45018b116b1729755c52
I20240502 01:58:29.305277 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 21624
I20240502 01:58:29.311870 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 21761
I20240502 01:58:29.317226 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 22828
I20240502 01:58:29.322139 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 22966
I20240502 01:58:29.326725 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 23102
I20240502 01:58:29.331187 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 23237
I20240502 01:58:29.335666 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 23373
I20240502 01:58:29.340085 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 25652
I20240502 01:58:29.344655 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 21530
2024-05-02T01:58:29Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/0 (29291 ms)
[----------] 1 test from RebalanceParamTest (29291 ms total)
[----------] 1 test from DDLDuringRebalancingTest
[ RUN ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0
I20240502 01:58:29.378139 19631 test_util.cc:268] Using random seed: -259694507
I20240502 01:58:29.378932 19631 ts_itest-base.cc:112] Starting cluster with:
I20240502 01:58:29.378969 19631 ts_itest-base.cc:113] --------------
I20240502 01:58:29.378979 19631 ts_itest-base.cc:114] 8 tablet servers
I20240502 01:58:29.378988 19631 ts_itest-base.cc:115] 3 replicas per TS
I20240502 01:58:29.378999 19631 ts_itest-base.cc:116] --------------
2024-05-02T01:58:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T01:58:29Z Disabled control of system clock
I20240502 01:58:29.386368 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:46361
--webserver_interface=127.19.43.254
--webserver_port=0
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.43.254:46361
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 01:58:29.487196 26484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:29.487406 26484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:29.487443 26484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:29.490916 26484 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:29.491067 26484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 01:58:29.491117 26484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:29.491153 26484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 01:58:29.491194 26484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 01:58:29.495287 26484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.19.43.254:46361
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:46361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.19.43.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:29.496062 26484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:29.497100 26484 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:29.503603 26492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:29.503667 26490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:29.503690 26489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:29.503960 26484 server_base.cc:1034] running on GCE node
I20240502 01:58:29.504382 26484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:29.505167 26484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:29.506301 26484 hybrid_clock.cc:648] HybridClock initialized: now 1714615109506294 us; error 27 us; skew 500 ppm
I20240502 01:58:29.508011 26484 webserver.cc:460] Webserver started at http://127.19.43.254:40195/ using document root <none> and password file <none>
I20240502 01:58:29.508500 26484 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:29.508551 26484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:29.508720 26484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:29.510305 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "35b243e102134ca6be77b68ec0929e02"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.510762 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "35b243e102134ca6be77b68ec0929e02"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.513698 26484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:29.515667 26498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.516556 26484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:29.516651 26484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
uuid: "35b243e102134ca6be77b68ec0929e02"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.516731 26484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:29.539511 26484 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:29.540118 26484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:29.540267 26484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:29.546530 26484 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.254:46361
I20240502 01:58:29.546537 26551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.254:46361 every 8 connection(s)
I20240502 01:58:29.547442 26484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 01:58:29.549818 26552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:29.550923 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 26484
I20240502 01:58:29.551048 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 01:58:29.555028 26552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Bootstrap starting.
I20240502 01:58:29.556996 26552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:29.557614 26552 log.cc:824] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:29.559197 26552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: No bootstrap required, opened a new log
I20240502 01:58:29.561573 26552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b243e102134ca6be77b68ec0929e02" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 46361 } }
I20240502 01:58:29.561733 26552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:29.561771 26552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b243e102134ca6be77b68ec0929e02, State: Initialized, Role: FOLLOWER
I20240502 01:58:29.562155 26552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b243e102134ca6be77b68ec0929e02" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 46361 } }
I20240502 01:58:29.562285 26552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:58:29.562347 26552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:58:29.562453 26552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:29.563102 26552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b243e102134ca6be77b68ec0929e02" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 46361 } }
I20240502 01:58:29.563385 26552 leader_election.cc:302] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35b243e102134ca6be77b68ec0929e02; no voters:
I20240502 01:58:29.563608 26552 leader_election.cc:288] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:58:29.563906 26558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:29.564074 26558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [term 1 LEADER]: Becoming Leader. State: Replica: 35b243e102134ca6be77b68ec0929e02, State: Running, Role: LEADER
I20240502 01:58:29.564325 26558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b243e102134ca6be77b68ec0929e02" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 46361 } }
I20240502 01:58:29.564545 26552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 01:58:29.565526 26560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35b243e102134ca6be77b68ec0929e02. Latest consensus state: current_term: 1 leader_uuid: "35b243e102134ca6be77b68ec0929e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b243e102134ca6be77b68ec0929e02" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 46361 } } }
I20240502 01:58:29.565629 26560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [sys.catalog]: This master's current role is: LEADER
I20240502 01:58:29.565783 26559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35b243e102134ca6be77b68ec0929e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b243e102134ca6be77b68ec0929e02" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 46361 } } }
I20240502 01:58:29.566087 26559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02 [sys.catalog]: This master's current role is: LEADER
W20240502 01:58:29.568845 26573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 01:58:29.568905 26573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 01:58:29.569015 26567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 01:58:29.569887 26567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 01:58:29.574142 26567 catalog_manager.cc:1347] Generated new cluster ID: 58df11c897cf4e8e8a16c991a57fcd8d
I20240502 01:58:29.574203 26567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 01:58:29.613308 26567 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 01:58:29.614584 26567 catalog_manager.cc:1504] Loading token signing keys...
I20240502 01:58:29.629381 26567 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Generated new TSK 0
I20240502 01:58:29.630223 26567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 01:58:29.640714 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.193:0
--local_ip_for_outbound_sockets=127.19.43.193
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:29.742913 26577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:29.743114 26577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:29.743172 26577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:29.746083 26577 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:29.746141 26577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:29.746224 26577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.193
I20240502 01:58:29.749627 26577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.26577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.193
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:29.750438 26577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:29.751485 26577 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 01:58:29.758672 26577 server_base.cc:1034] running on GCE node
W20240502 01:58:29.758591 26583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:29.758608 26585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:29.758625 26582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:29.759160 26577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:29.759644 26577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:29.760797 26577 hybrid_clock.cc:648] HybridClock initialized: now 1714615109760785 us; error 25 us; skew 500 ppm
I20240502 01:58:29.762379 26577 webserver.cc:460] Webserver started at http://127.19.43.193:37023/ using document root <none> and password file <none>
I20240502 01:58:29.762864 26577 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:29.762917 26577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:29.763088 26577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:29.764590 26577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "998b57e2badc4d779c586c6579149114"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.765048 26577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "998b57e2badc4d779c586c6579149114"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.768131 26577 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:58:29.769995 26591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.770905 26577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:29.771005 26577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "998b57e2badc4d779c586c6579149114"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.771080 26577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:29.791082 26577 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:29.791764 26577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:29.791932 26577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:29.792464 26577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:29.793269 26577 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:29.793323 26577 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.793359 26577 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:29.793378 26577 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.801113 26577 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.193:32929
I20240502 01:58:29.801151 26707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.193:32929 every 8 connection(s)
I20240502 01:58:29.801936 26577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 01:58:29.806006 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 26577
I20240502 01:58:29.806131 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 01:58:29.808140 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.194:0
--local_ip_for_outbound_sockets=127.19.43.194
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:29.810076 26708 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:29.810271 26708 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:29.810654 26708 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:29.811983 26515 ts_manager.cc:194] Registered new tserver with Master: 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:29.813539 26515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.193:49407
W20240502 01:58:29.908803 26711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:29.908969 26711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:29.909008 26711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:29.911907 26711 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:29.911962 26711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:29.912021 26711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.194
I20240502 01:58:29.915369 26711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.26711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.194
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:29.916018 26711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:29.916940 26711 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:29.923728 26720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:29.923723 26717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:29.923727 26718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:29.923975 26711 server_base.cc:1034] running on GCE node
I20240502 01:58:29.924276 26711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:29.924743 26711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:29.925900 26711 hybrid_clock.cc:648] HybridClock initialized: now 1714615109925886 us; error 25 us; skew 500 ppm
I20240502 01:58:29.927491 26711 webserver.cc:460] Webserver started at http://127.19.43.194:37665/ using document root <none> and password file <none>
I20240502 01:58:29.927953 26711 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:29.928007 26711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:29.928155 26711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:29.929548 26711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "b290c1bf893444078ed721f25995d124"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.929958 26711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "b290c1bf893444078ed721f25995d124"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.932817 26711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:29.934754 26726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.935604 26711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:29.935698 26711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "b290c1bf893444078ed721f25995d124"
format_stamp: "Formatted at 2024-05-02 01:58:29 on dist-test-slave-mnd3"
I20240502 01:58:29.935771 26711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:29.947521 26711 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:29.948014 26711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:29.948143 26711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:29.948581 26711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:29.949345 26711 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:29.949389 26711 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.949434 26711 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:29.949462 26711 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:29.957159 26711 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.194:38007
I20240502 01:58:29.957204 26840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.194:38007 every 8 connection(s)
I20240502 01:58:29.957909 26711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 01:58:29.964066 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 26711
I20240502 01:58:29.964181 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 01:58:29.965274 26841 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:29.965459 26841 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:29.965885 26841 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:29.966255 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.195:0
--local_ip_for_outbound_sockets=127.19.43.195
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:29.966763 26514 ts_manager.cc:194] Registered new tserver with Master: b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:29.967295 26514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.194:39465
W20240502 01:58:30.066190 26844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:30.066366 26844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:30.066416 26844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:30.069285 26844 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:30.069344 26844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:30.069422 26844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.195
I20240502 01:58:30.072820 26844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.26844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.195
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:30.073529 26844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:30.074561 26844 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:30.081686 26850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.081658 26852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.081669 26849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.081918 26844 server_base.cc:1034] running on GCE node
I20240502 01:58:30.082250 26844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:30.082729 26844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:30.083878 26844 hybrid_clock.cc:648] HybridClock initialized: now 1714615110083867 us; error 30 us; skew 500 ppm
I20240502 01:58:30.085460 26844 webserver.cc:460] Webserver started at http://127.19.43.195:42107/ using document root <none> and password file <none>
I20240502 01:58:30.085940 26844 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:30.085995 26844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:30.086185 26844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:30.087668 26844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "842c1b011b5d4c77bfd9b9591111502e"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.088089 26844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "842c1b011b5d4c77bfd9b9591111502e"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.090981 26844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 01:58:30.092854 26858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.093720 26844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:30.093818 26844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "842c1b011b5d4c77bfd9b9591111502e"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.093892 26844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:30.113119 26844 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:30.113610 26844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:30.113739 26844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:30.114212 26844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:30.114969 26844 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:30.115013 26844 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.115059 26844 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:30.115087 26844 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.123046 26844 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.195:40379
I20240502 01:58:30.123077 26976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.195:40379 every 8 connection(s)
I20240502 01:58:30.123793 26844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 01:58:30.131068 26977 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:30.131248 26977 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:30.131603 26977 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:30.131633 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 26844
I20240502 01:58:30.131736 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 01:58:30.132490 26514 ts_manager.cc:194] Registered new tserver with Master: 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:30.133100 26514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.195:50557
I20240502 01:58:30.133478 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.196:0
--local_ip_for_outbound_sockets=127.19.43.196
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:30.231132 26980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:30.231299 26980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:30.231338 26980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:30.234158 26980 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:30.234206 26980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:30.234272 26980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.196
I20240502 01:58:30.237571 26980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.26980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.196
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:30.238255 26980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:30.239215 26980 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:30.245893 26988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.245925 26985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.245901 26986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.246106 26980 server_base.cc:1034] running on GCE node
I20240502 01:58:30.246490 26980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:30.246919 26980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:30.248055 26980 hybrid_clock.cc:648] HybridClock initialized: now 1714615110248034 us; error 32 us; skew 500 ppm
I20240502 01:58:30.249594 26980 webserver.cc:460] Webserver started at http://127.19.43.196:41163/ using document root <none> and password file <none>
I20240502 01:58:30.250094 26980 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:30.250149 26980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:30.250325 26980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:30.251767 26980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "68e5b1eeb04448fea818a14991eab87c"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.252182 26980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "68e5b1eeb04448fea818a14991eab87c"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.255060 26980 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 01:58:30.256892 26994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.257820 26980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:30.257912 26980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "68e5b1eeb04448fea818a14991eab87c"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.257987 26980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:30.270571 26980 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:30.271062 26980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:30.271188 26980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:30.271627 26980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:30.272373 26980 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:30.272418 26980 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.272464 26980 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:30.272491 26980 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.280472 26980 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.196:43635
I20240502 01:58:30.280493 27109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.196:43635 every 8 connection(s)
I20240502 01:58:30.281220 26980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 01:58:30.288228 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 26980
I20240502 01:58:30.288321 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 01:58:30.288447 27110 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:30.288647 27110 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:30.289079 27110 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:30.289911 26514 ts_manager.cc:194] Registered new tserver with Master: 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:30.290232 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:0
--local_ip_for_outbound_sockets=127.19.43.197
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:30.290469 26514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.196:56251
W20240502 01:58:30.387738 27113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:30.387907 27113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:30.387944 27113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:30.390746 27113 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:30.390795 27113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:30.390856 27113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:58:30.394169 27113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.27113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:30.394809 27113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:30.395890 27113 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:30.402669 27118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.402707 27121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.402678 27119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.402892 27113 server_base.cc:1034] running on GCE node
I20240502 01:58:30.403249 27113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:30.403700 27113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:30.404837 27113 hybrid_clock.cc:648] HybridClock initialized: now 1714615110404822 us; error 28 us; skew 500 ppm
I20240502 01:58:30.406410 27113 webserver.cc:460] Webserver started at http://127.19.43.197:38069/ using document root <none> and password file <none>
I20240502 01:58:30.406868 27113 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:30.406922 27113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:30.407073 27113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:30.408496 27113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "f3d18a019013487ab801599cfb80e4b4"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.408895 27113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "f3d18a019013487ab801599cfb80e4b4"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.411738 27113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:58:30.413537 27127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.414407 27113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:30.414497 27113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "f3d18a019013487ab801599cfb80e4b4"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.414567 27113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:30.441447 27113 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:30.441949 27113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:30.442104 27113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:30.442564 27113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:30.443331 27113 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:30.443374 27113 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.443423 27113 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:30.443449 27113 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.451063 27113 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:34601
I20240502 01:58:30.451093 27243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:34601 every 8 connection(s)
I20240502 01:58:30.451823 27113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:58:30.454702 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 27113
I20240502 01:58:30.454798 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 01:58:30.456935 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.198:0
--local_ip_for_outbound_sockets=127.19.43.198
--webserver_interface=127.19.43.198
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:30.459921 27244 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:30.460137 27244 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:30.460631 27244 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:30.461588 26514 ts_manager.cc:194] Registered new tserver with Master: f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:30.462141 26514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:40097
W20240502 01:58:30.555130 27247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:30.555316 27247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:30.555374 27247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:30.558267 27247 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:30.558326 27247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:30.558403 27247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.198
I20240502 01:58:30.561749 27247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.19.43.198
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.27247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.198
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:30.562449 27247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:30.563356 27247 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:30.570152 27253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.570153 27252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.570149 27255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.570305 27247 server_base.cc:1034] running on GCE node
I20240502 01:58:30.570716 27247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:30.571182 27247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:30.572319 27247 hybrid_clock.cc:648] HybridClock initialized: now 1714615110572306 us; error 24 us; skew 500 ppm
I20240502 01:58:30.573951 27247 webserver.cc:460] Webserver started at http://127.19.43.198:36827/ using document root <none> and password file <none>
I20240502 01:58:30.574472 27247 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:30.574525 27247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:30.574683 27247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:30.576514 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.577051 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.580968 27247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20240502 01:58:30.583238 27261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.584141 27247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:30.584229 27247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.584306 27247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:30.602420 27247 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:30.602936 27247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:30.603062 27247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:30.603502 27247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:30.604290 27247 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:30.604332 27247 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.604370 27247 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:30.604398 27247 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.611976 27247 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.198:40103
I20240502 01:58:30.612004 27376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.198:40103 every 8 connection(s)
I20240502 01:58:30.612740 27247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 01:58:30.620046 27377 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:30.620223 27377 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:30.620600 27377 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:30.621400 26514 ts_manager.cc:194] Registered new tserver with Master: 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:30.621788 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 27247
I20240502 01:58:30.621884 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 01:58:30.621932 26514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.198:51503
I20240502 01:58:30.623807 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.199:0
--local_ip_for_outbound_sockets=127.19.43.199
--webserver_interface=127.19.43.199
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:30.721324 27380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:30.721487 27380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:30.721526 27380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:30.724426 27380 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:30.724478 27380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:30.724539 27380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.199
I20240502 01:58:30.727896 27380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.19.43.199
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.27380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.199
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:30.728549 27380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:30.729444 27380 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:30.736100 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.736119 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.736254 27380 server_base.cc:1034] running on GCE node
W20240502 01:58:30.736117 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.736653 27380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:30.737110 27380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:30.738250 27380 hybrid_clock.cc:648] HybridClock initialized: now 1714615110738239 us; error 24 us; skew 500 ppm
I20240502 01:58:30.739814 27380 webserver.cc:460] Webserver started at http://127.19.43.199:35609/ using document root <none> and password file <none>
I20240502 01:58:30.740272 27380 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:30.740334 27380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:30.740484 27380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:30.741884 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "468dd27456c64d67b22e8a58af93d93d"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.742339 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "468dd27456c64d67b22e8a58af93d93d"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.745110 27380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:30.746978 27394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.747882 27380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:30.747980 27380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "468dd27456c64d67b22e8a58af93d93d"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.748051 27380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:30.776093 27380 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:30.776584 27380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:30.776710 27380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:30.777143 27380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:30.777925 27380 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:30.777968 27380 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.778015 27380 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:30.778066 27380 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.785828 27380 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.199:44303
I20240502 01:58:30.785859 27509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.199:44303 every 8 connection(s)
I20240502 01:58:30.786656 27380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 01:58:30.788173 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 27380
I20240502 01:58:30.788277 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 01:58:30.790253 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.200:0
--local_ip_for_outbound_sockets=127.19.43.200
--webserver_interface=127.19.43.200
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:30.794814 27510 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:30.795013 27510 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:30.795403 27510 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:30.796269 26514 ts_manager.cc:194] Registered new tserver with Master: 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:30.796780 26514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.199:55323
I20240502 01:58:30.815524 26708 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
W20240502 01:58:30.889421 27513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:30.889592 27513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:30.889650 27513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:30.892510 27513 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:30.892566 27513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:30.892647 27513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.200
I20240502 01:58:30.895992 27513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.200:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.19.43.200
--webserver_port=0
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.27513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.200
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:30.896703 27513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:30.897637 27513 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:30.904608 27520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:30.904690 27518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.904707 27513 server_base.cc:1034] running on GCE node
W20240502 01:58:30.904608 27522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:30.905133 27513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:30.905582 27513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:30.906725 27513 hybrid_clock.cc:648] HybridClock initialized: now 1714615110906711 us; error 24 us; skew 500 ppm
I20240502 01:58:30.908330 27513 webserver.cc:460] Webserver started at http://127.19.43.200:35665/ using document root <none> and password file <none>
I20240502 01:58:30.908793 27513 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:30.908843 27513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:30.908999 27513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:58:30.910461 27513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "7c693d0933c64d9babc07f404df45cd1"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.910866 27513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "7c693d0933c64d9babc07f404df45cd1"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.913686 27513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:30.915572 27528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.916435 27513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:30.916534 27513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "7c693d0933c64d9babc07f404df45cd1"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:30.916604 27513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:30.937011 27513 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:30.937498 27513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:30.937623 27513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:30.938093 27513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:30.938835 27513 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:30.938879 27513 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.938925 27513 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:30.938951 27513 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:30.946449 27513 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.200:36259
I20240502 01:58:30.946477 27642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.200:36259 every 8 connection(s)
I20240502 01:58:30.947178 27513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 01:58:30.954702 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 27513
I20240502 01:58:30.954826 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 01:58:30.954797 27643 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:30.954982 27643 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:30.955376 27643 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:30.956398 26515 ts_manager.cc:194] Registered new tserver with Master: 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:30.957043 26515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.200:35353
I20240502 01:58:30.968384 19631 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 01:58:30.968984 26841 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:30.976842 19631 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 01:58:30.976907 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 27113
I20240502 01:58:30.982446 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 27247
I20240502 01:58:30.987737 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 27380
I20240502 01:58:30.992420 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 27513
I20240502 01:58:31.135015 26977 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:31.292230 27110 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:34.748412 26514 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:34.767786 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 7680bb0330234d0ca29b507a6e329b78 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.767786 26909 tablet_service.cc:1467] Processing CreateTablet for tablet e2a7f26d772f43179e473985bc8997d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.767877 26775 tablet_service.cc:1467] Processing CreateTablet for tablet e2a7f26d772f43179e473985bc8997d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.767889 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 669e26634aaf4fa59df4a3c44772f3a5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.768716 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7680bb0330234d0ca29b507a6e329b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.768716 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a7f26d772f43179e473985bc8997d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.768729 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a7f26d772f43179e473985bc8997d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.769016 27044 tablet_service.cc:1467] Processing CreateTablet for tablet e2a7f26d772f43179e473985bc8997d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.769270 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2a7f26d772f43179e473985bc8997d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.770128 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 669e26634aaf4fa59df4a3c44772f3a5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.770337 26773 tablet_service.cc:1467] Processing CreateTablet for tablet 7680bb0330234d0ca29b507a6e329b78 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.770398 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669e26634aaf4fa59df4a3c44772f3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.770645 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7680bb0330234d0ca29b507a6e329b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.771019 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669e26634aaf4fa59df4a3c44772f3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.772663 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 669e26634aaf4fa59df4a3c44772f3a5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.773402 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669e26634aaf4fa59df4a3c44772f3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.777073 27663 tablet_bootstrap.cc:492] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:34.778236 27664 tablet_bootstrap.cc:492] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:34.778275 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 7680bb0330234d0ca29b507a6e329b78 (DEFAULT_TABLE table=rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:34.778543 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7680bb0330234d0ca29b507a6e329b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:34.779403 27663 tablet_bootstrap.cc:654] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.779450 27665 tablet_bootstrap.cc:492] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:34.780248 27663 log.cc:824] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:34.780342 27664 tablet_bootstrap.cc:654] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.781045 27664 log.cc:824] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:34.781237 27665 tablet_bootstrap.cc:654] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.781950 27663 tablet_bootstrap.cc:492] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:34.781963 27665 log.cc:824] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:34.782128 27663 ts_tablet_manager.cc:1367] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 01:58:34.782358 27666 tablet_bootstrap.cc:492] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:34.782649 27664 tablet_bootstrap.cc:492] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:34.782806 27664 ts_tablet_manager.cc:1367] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 01:58:34.783658 27665 tablet_bootstrap.cc:492] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:34.783798 27665 ts_tablet_manager.cc:1367] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:58:34.784123 27666 tablet_bootstrap.cc:654] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.784962 27666 log.cc:824] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:34.785698 27663 raft_consensus.cc:348] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.785898 27663 raft_consensus.cc:374] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.785945 27663 raft_consensus.cc:729] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.786290 27664 raft_consensus.cc:348] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.786468 27664 raft_consensus.cc:374] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.786403 27663 consensus_queue.cc:260] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.786518 27664 raft_consensus.cc:729] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.786924 27664 consensus_queue.cc:260] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.787256 27663 ts_tablet_manager.cc:1398] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 01:58:34.787176 27665 raft_consensus.cc:348] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.787343 27665 raft_consensus.cc:374] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.787389 27665 raft_consensus.cc:729] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.787581 27663 tablet_bootstrap.cc:492] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:34.787685 27666 tablet_bootstrap.cc:492] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:34.787709 27664 ts_tablet_manager.cc:1398] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 01:58:34.787827 27666 ts_tablet_manager.cc:1367] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240502 01:58:34.788012 27664 tablet_bootstrap.cc:492] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:34.788439 27663 tablet_bootstrap.cc:654] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.788846 27664 tablet_bootstrap.cc:654] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.789748 27663 tablet_bootstrap.cc:492] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:34.789780 27664 tablet_bootstrap.cc:492] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:34.789819 27663 ts_tablet_manager.cc:1367] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:34.789837 27664 ts_tablet_manager.cc:1367] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:34.790323 27663 raft_consensus.cc:348] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.790323 27664 raft_consensus.cc:348] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.790408 27664 raft_consensus.cc:374] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.790408 27663 raft_consensus.cc:374] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.790438 27663 raft_consensus.cc:729] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.790438 27664 raft_consensus.cc:729] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.790541 27664 consensus_queue.cc:260] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.790541 27663 consensus_queue.cc:260] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.790771 27663 ts_tablet_manager.cc:1398] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:34.790777 27664 ts_tablet_manager.cc:1398] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:34.791241 27666 raft_consensus.cc:348] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.791296 27665 consensus_queue.cc:260] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.791766 27666 raft_consensus.cc:374] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.791832 27666 raft_consensus.cc:729] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.792078 27665 ts_tablet_manager.cc:1398] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240502 01:58:34.792735 27666 consensus_queue.cc:260] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.793005 27665 tablet_bootstrap.cc:492] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:34.793498 27666 ts_tablet_manager.cc:1398] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 01:58:34.793867 27665 tablet_bootstrap.cc:654] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.794500 27666 tablet_bootstrap.cc:492] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:34.794963 27665 tablet_bootstrap.cc:492] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:34.795086 27665 ts_tablet_manager.cc:1367] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:34.795364 27666 tablet_bootstrap.cc:654] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.795696 27665 raft_consensus.cc:348] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.795845 27665 raft_consensus.cc:374] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.795902 27665 raft_consensus.cc:729] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.796015 27665 consensus_queue.cc:260] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.796263 27665 ts_tablet_manager.cc:1398] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:34.796451 27665 tablet_bootstrap.cc:492] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:34.796494 27666 tablet_bootstrap.cc:492] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:34.796554 27666 ts_tablet_manager.cc:1367] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:34.797031 27666 raft_consensus.cc:348] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.797123 27666 raft_consensus.cc:374] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.797153 27666 raft_consensus.cc:729] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.797291 27666 consensus_queue.cc:260] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.797343 27665 tablet_bootstrap.cc:654] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:34.797626 27666 ts_tablet_manager.cc:1398] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:34.798483 27665 tablet_bootstrap.cc:492] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:34.798573 27665 ts_tablet_manager.cc:1367] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:34.798983 27665 raft_consensus.cc:348] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.799100 27665 raft_consensus.cc:374] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:34.799139 27665 raft_consensus.cc:729] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:34.799249 27665 consensus_queue.cc:260] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.799448 27665 ts_tablet_manager.cc:1398] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 01:58:34.805570 26709 tablet.cc:2369] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:34.817276 27675 raft_consensus.cc:482] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:34.817425 27675 raft_consensus.cc:504] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.818296 27675 leader_election.cc:288] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:34.821843 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680bb0330234d0ca29b507a6e329b78" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:34.821957 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680bb0330234d0ca29b507a6e329b78" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:34.822081 26661 raft_consensus.cc:2454] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:34.822193 27064 raft_consensus.cc:2454] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:34.822556 26730 leader_election.cc:302] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:34.822762 27675 raft_consensus.cc:2781] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:34.822830 27675 raft_consensus.cc:482] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:34.822868 27675 raft_consensus.cc:3037] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.823580 27675 raft_consensus.cc:504] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.823854 27675 leader_election.cc:288] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:34.824138 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680bb0330234d0ca29b507a6e329b78" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:34.824167 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7680bb0330234d0ca29b507a6e329b78" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:34.824240 26661 raft_consensus.cc:3037] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.824261 27064 raft_consensus.cc:3037] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.824950 26661 raft_consensus.cc:2454] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:34.825055 27064 raft_consensus.cc:2454] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:34.825234 26729 leader_election.cc:302] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:34.825415 27675 raft_consensus.cc:2781] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:34.825583 27675 raft_consensus.cc:686] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:34.825811 27675 consensus_queue.cc:237] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:34.828131 26513 catalog_manager.cc:5484] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:34.867460 27672 raft_consensus.cc:482] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:34.867611 27672 raft_consensus.cc:504] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.868536 27672 leader_election.cc:288] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:34.871660 27676 raft_consensus.cc:482] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:34.871769 27676 raft_consensus.cc:504] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.872022 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a7f26d772f43179e473985bc8997d5" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:34.872148 27064 raft_consensus.cc:2454] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:34.872373 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a7f26d772f43179e473985bc8997d5" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:34.872457 26862 leader_election.cc:302] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:34.872534 26795 raft_consensus.cc:2454] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:34.872658 27672 raft_consensus.cc:2781] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:34.872721 27672 raft_consensus.cc:482] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:34.872758 27672 raft_consensus.cc:3037] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.872756 27676 leader_election.cc:288] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:34.873775 27672 raft_consensus.cc:504] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.874145 27672 leader_election.cc:288] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:34.874511 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a7f26d772f43179e473985bc8997d5" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:34.874588 27064 raft_consensus.cc:3037] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.875195 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2a7f26d772f43179e473985bc8997d5" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:34.875301 26795 raft_consensus.cc:3037] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.875423 27064 raft_consensus.cc:2454] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:34.875728 26862 leader_election.cc:302] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:34.875928 27672 raft_consensus.cc:2781] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:34.876099 27672 raft_consensus.cc:686] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:34.876251 26795 raft_consensus.cc:2454] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:34.876387 27672 consensus_queue.cc:237] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.877241 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669e26634aaf4fa59df4a3c44772f3a5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
W20240502 01:58:34.877295 26978 tablet.cc:2369] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:34.877333 26795 raft_consensus.cc:2454] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:34.877684 26594 leader_election.cc:302] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:34.877889 27676 raft_consensus.cc:2781] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:34.877948 27676 raft_consensus.cc:482] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:34.877988 27676 raft_consensus.cc:3037] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.878743 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669e26634aaf4fa59df4a3c44772f3a5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:34.878894 26930 raft_consensus.cc:2454] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:34.878863 27676 raft_consensus.cc:504] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.879184 27676 leader_election.cc:288] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:34.879662 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669e26634aaf4fa59df4a3c44772f3a5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:34.879662 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669e26634aaf4fa59df4a3c44772f3a5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:34.879765 26795 raft_consensus.cc:3037] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.879765 26930 raft_consensus.cc:3037] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:34.880223 26513 catalog_manager.cc:5484] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:34.880543 26795 raft_consensus.cc:2454] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:34.880558 26930 raft_consensus.cc:2454] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:34.880832 26594 leader_election.cc:302] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:34.881072 27676 raft_consensus.cc:2781] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:34.881269 27676 raft_consensus.cc:686] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:34.881528 27676 consensus_queue.cc:237] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:34.883891 26513 catalog_manager.cc:5484] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:34.961720 26842 tablet.cc:2369] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:58:34.987967 27693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:34.988153 27693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:34.991063 27693 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
W20240502 01:58:35.034763 27111 tablet.cc:2369] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:35.070814 26795 raft_consensus.cc:1261] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.071005 27064 raft_consensus.cc:1261] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.071381 27689 consensus_queue.cc:1035] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.071653 27672 consensus_queue.cc:1035] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.073251 26930 raft_consensus.cc:1261] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.073784 27676 consensus_queue.cc:1035] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.075445 26794 raft_consensus.cc:1261] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.075919 27691 consensus_queue.cc:1035] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.079183 27725 mvcc.cc:205] Tried to move back new op lower bound from 7023063511332478976 to 7023063510556971008. Current Snapshot: MvccSnapshot[applied={T|T < 7023063511332478976}]
I20240502 01:58:35.082437 27063 raft_consensus.cc:1261] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.082883 27675 consensus_queue.cc:1035] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.083035 26661 raft_consensus.cc:1261] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.083482 27675 consensus_queue.cc:1035] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.087569 27733 mvcc.cc:205] Tried to move back new op lower bound from 7023063511332478976 to 7023063510556971008. Current Snapshot: MvccSnapshot[applied={T|T < 7023063511332478976}]
I20240502 01:58:35.092824 27724 mvcc.cc:205] Tried to move back new op lower bound from 7023063511371333632 to 7023063510536114176. Current Snapshot: MvccSnapshot[applied={T|T < 7023063511371333632}]
I20240502 01:58:35.098779 27729 mvcc.cc:205] Tried to move back new op lower bound from 7023063511323676672 to 7023063510536114176. Current Snapshot: MvccSnapshot[applied={T|T < 7023063511323676672}]
INSERT report
rows total: 2000
time total: 114.846 ms
time per row: 0.0574229 ms
I20240502 01:58:35.142704 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:35.147526 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 5515469fdf494900ae20f20fdfca587c (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.147616 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 5515469fdf494900ae20f20fdfca587c (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.147717 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 5515469fdf494900ae20f20fdfca587c (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.147828 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5515469fdf494900ae20f20fdfca587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.147879 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5515469fdf494900ae20f20fdfca587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.147924 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5515469fdf494900ae20f20fdfca587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.148567 26640 tablet_service.cc:1467] Processing CreateTablet for tablet dfdd6d487b57479cb48a92bc770947ce (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.148774 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdd6d487b57479cb48a92bc770947ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.149847 27663 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.149997 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 50bfc86904bc4347bd6beb7f16eb638d (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.150239 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bfc86904bc4347bd6beb7f16eb638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.150494 27666 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.150672 27663 tablet_bootstrap.cc:654] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.151136 27666 tablet_bootstrap.cc:654] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.151413 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 50bfc86904bc4347bd6beb7f16eb638d (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.151641 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bfc86904bc4347bd6beb7f16eb638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.152070 27664 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.152487 27666 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.152545 27666 ts_tablet_manager.cc:1367] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.152779 27664 tablet_bootstrap.cc:654] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.152861 27666 raft_consensus.cc:348] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.152933 27666 raft_consensus.cc:374] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.152951 27666 raft_consensus.cc:729] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.153051 27666 consensus_queue.cc:260] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.153293 27666 ts_tablet_manager.cc:1398] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:35.153582 27665 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:35.154052 27043 tablet_service.cc:1467] Processing CreateTablet for tablet dfdd6d487b57479cb48a92bc770947ce (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.154263 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdd6d487b57479cb48a92bc770947ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.154383 27663 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.154454 27663 ts_tablet_manager.cc:1367] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:35.154592 27665 tablet_bootstrap.cc:654] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.154552 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 50bfc86904bc4347bd6beb7f16eb638d (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.154747 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bfc86904bc4347bd6beb7f16eb638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.154878 27663 raft_consensus.cc:348] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.154968 27663 raft_consensus.cc:374] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.154995 27663 raft_consensus.cc:729] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.155110 27663 consensus_queue.cc:260] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.155337 27663 ts_tablet_manager.cc:1398] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:35.155666 27666 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.156466 27666 tablet_bootstrap.cc:654] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.156672 27663 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.156709 26909 tablet_service.cc:1467] Processing CreateTablet for tablet dfdd6d487b57479cb48a92bc770947ce (DEFAULT_TABLE table=rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.156909 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdd6d487b57479cb48a92bc770947ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.157424 27663 tablet_bootstrap.cc:654] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.158479 27666 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.158555 27666 ts_tablet_manager.cc:1367] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:35.158953 27666 raft_consensus.cc:348] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.159049 27666 raft_consensus.cc:374] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.159075 27664 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.159081 27666 raft_consensus.cc:729] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.159138 27664 ts_tablet_manager.cc:1367] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:35.159186 27666 consensus_queue.cc:260] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.159432 27666 ts_tablet_manager.cc:1398] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.159577 27666 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.159528 27664 raft_consensus.cc:348] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.159613 27664 raft_consensus.cc:374] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.159641 27664 raft_consensus.cc:729] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.159745 27664 consensus_queue.cc:260] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.159981 27664 ts_tablet_manager.cc:1398] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.160077 27663 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.160125 27664 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.160161 27663 ts_tablet_manager.cc:1367] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:35.160372 27666 tablet_bootstrap.cc:654] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.160569 27663 raft_consensus.cc:348] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.160676 27665 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:35.160687 27663 raft_consensus.cc:374] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.160722 27663 raft_consensus.cc:729] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.160773 27665 ts_tablet_manager.cc:1367] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:35.160817 27663 consensus_queue.cc:260] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.160902 27664 tablet_bootstrap.cc:654] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.161065 27663 ts_tablet_manager.cc:1398] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.161154 27665 raft_consensus.cc:348] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.161257 27665 raft_consensus.cc:374] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.161288 27665 raft_consensus.cc:729] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.161392 27665 consensus_queue.cc:260] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.161495 27666 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.161564 27666 ts_tablet_manager.cc:1367] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.161631 27665 ts_tablet_manager.cc:1398] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.161954 27666 raft_consensus.cc:348] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.162199 27666 raft_consensus.cc:374] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.162256 27666 raft_consensus.cc:729] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.162369 27666 consensus_queue.cc:260] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.162606 27666 ts_tablet_manager.cc:1398] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.163038 27664 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.163086 27664 ts_tablet_manager.cc:1367] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:35.163416 27664 raft_consensus.cc:348] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.163504 27664 raft_consensus.cc:374] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.163533 27664 raft_consensus.cc:729] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.163625 27664 consensus_queue.cc:260] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.163870 27664 ts_tablet_manager.cc:1398] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.164006 27664 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.164649 27664 tablet_bootstrap.cc:654] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.165582 27664 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.165655 27664 ts_tablet_manager.cc:1367] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.166005 27664 raft_consensus.cc:348] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.166127 27664 raft_consensus.cc:374] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.166165 27664 raft_consensus.cc:729] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.166270 27664 consensus_queue.cc:260] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.166512 27664 ts_tablet_manager.cc:1398] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.219612 27671 raft_consensus.cc:482] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:35.219718 27671 raft_consensus.cc:504] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.220695 27671 leader_election.cc:288] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:35.225088 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdd6d487b57479cb48a92bc770947ce" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:35.225088 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdd6d487b57479cb48a92bc770947ce" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:35.225222 26930 raft_consensus.cc:2454] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:35.225222 26661 raft_consensus.cc:2454] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:35.225539 26998 leader_election.cc:302] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:35.225765 27671 raft_consensus.cc:2781] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:35.225824 27671 raft_consensus.cc:482] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:35.225849 27671 raft_consensus.cc:3037] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.226624 27671 raft_consensus.cc:504] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.226943 27671 leader_election.cc:288] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:35.227221 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdd6d487b57479cb48a92bc770947ce" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:35.227238 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdd6d487b57479cb48a92bc770947ce" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:35.227339 26661 raft_consensus.cc:3037] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.227339 26930 raft_consensus.cc:3037] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.228217 26661 raft_consensus.cc:2454] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:35.228217 26930 raft_consensus.cc:2454] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:35.228477 26998 leader_election.cc:302] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:35.228634 27671 raft_consensus.cc:2781] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:35.228825 27671 raft_consensus.cc:686] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:35.229111 27671 consensus_queue.cc:237] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.230046 27672 raft_consensus.cc:482] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:35.230126 27672 raft_consensus.cc:504] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.230369 27691 raft_consensus.cc:482] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:35.230458 27691 raft_consensus.cc:504] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.230548 27672 leader_election.cc:288] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:35.230787 27691 leader_election.cc:288] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:35.230877 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5515469fdf494900ae20f20fdfca587c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:35.230981 27063 raft_consensus.cc:2454] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:35.231168 26512 catalog_manager.cc:5484] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:35.231344 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bfc86904bc4347bd6beb7f16eb638d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:35.231473 26795 raft_consensus.cc:2454] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:35.231747 26862 leader_election.cc:302] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:35.231776 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bfc86904bc4347bd6beb7f16eb638d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:35.231871 26930 raft_consensus.cc:2454] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:35.231905 27672 raft_consensus.cc:2781] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:35.231959 27672 raft_consensus.cc:482] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:35.231990 27672 raft_consensus.cc:3037] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.232043 26594 leader_election.cc:302] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:35.232713 27672 raft_consensus.cc:504] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.233026 27691 raft_consensus.cc:2781] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:35.233037 27672 leader_election.cc:288] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:35.233084 27691 raft_consensus.cc:482] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:35.233136 27691 raft_consensus.cc:3037] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.234046 27691 raft_consensus.cc:504] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.234346 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5515469fdf494900ae20f20fdfca587c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:35.234369 27691 leader_election.cc:288] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:35.234444 27063 raft_consensus.cc:3037] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.234810 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bfc86904bc4347bd6beb7f16eb638d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:35.234907 26795 raft_consensus.cc:3037] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.235112 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bfc86904bc4347bd6beb7f16eb638d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:35.235208 26930 raft_consensus.cc:3037] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.235251 27063 raft_consensus.cc:2454] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:35.235561 26862 leader_election.cc:302] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:35.235703 27672 raft_consensus.cc:2781] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:35.235771 27672 raft_consensus.cc:686] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:35.235813 26795 raft_consensus.cc:2454] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:35.235893 26930 raft_consensus.cc:2454] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:35.235894 27672 consensus_queue.cc:237] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.236131 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5515469fdf494900ae20f20fdfca587c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:35.236187 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5515469fdf494900ae20f20fdfca587c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:35.236254 26661 raft_consensus.cc:2454] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:35.236526 26595 leader_election.cc:302] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:35.236667 27691 raft_consensus.cc:2781] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:35.236758 27691 raft_consensus.cc:686] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:35.236882 27691 consensus_queue.cc:237] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.237483 26512 catalog_manager.cc:5484] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:35.238298 26508 catalog_manager.cc:5484] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:35.353281 27767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:35.353466 27767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:35.356360 27767 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 01:58:35.429239 26795 raft_consensus.cc:1261] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.429672 27676 consensus_queue.cc:1035] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.431704 26930 raft_consensus.cc:1261] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.432986 26930 raft_consensus.cc:1261] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.433399 27063 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.433440 27763 consensus_queue.cc:1035] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.433794 26661 raft_consensus.cc:1261] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.434381 26661 raft_consensus.cc:3037] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.434643 27676 consensus_queue.cc:1035] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.435070 27763 consensus_queue.cc:1035] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.435366 26661 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.435988 27689 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.447619 27672 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 97.3112 ms
time per row: 0.0486556 ms
I20240502 01:58:35.490274 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:35.496364 26639 tablet_service.cc:1467] Processing CreateTablet for tablet a42923ea9b5f438a8760fcff8b93e288 (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.496484 26775 tablet_service.cc:1467] Processing CreateTablet for tablet a42923ea9b5f438a8760fcff8b93e288 (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.496546 27043 tablet_service.cc:1467] Processing CreateTablet for tablet a42923ea9b5f438a8760fcff8b93e288 (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.496660 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42923ea9b5f438a8760fcff8b93e288. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.496747 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42923ea9b5f438a8760fcff8b93e288. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.496766 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42923ea9b5f438a8760fcff8b93e288. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.497864 26640 tablet_service.cc:1467] Processing CreateTablet for tablet c618121cd88d4a53a6badd33b99c439c (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.498152 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c618121cd88d4a53a6badd33b99c439c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.498723 27665 tablet_bootstrap.cc:492] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:35.498764 27663 tablet_bootstrap.cc:492] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.498871 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 19295fe971e7409384b39937ea5d80e3 (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.499092 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19295fe971e7409384b39937ea5d80e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.499526 27665 tablet_bootstrap.cc:654] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.499560 27663 tablet_bootstrap.cc:654] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.500154 27666 tablet_bootstrap.cc:492] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.500960 27666 tablet_bootstrap.cc:654] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.500918 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 19295fe971e7409384b39937ea5d80e3 (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.501045 27664 tablet_bootstrap.cc:492] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.501114 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19295fe971e7409384b39937ea5d80e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.501207 27044 tablet_service.cc:1467] Processing CreateTablet for tablet c618121cd88d4a53a6badd33b99c439c (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.501425 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c618121cd88d4a53a6badd33b99c439c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.501827 27664 tablet_bootstrap.cc:654] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.501870 27666 tablet_bootstrap.cc:492] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.501935 27666 ts_tablet_manager.cc:1367] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.502445 27666 raft_consensus.cc:348] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.502539 27666 raft_consensus.cc:374] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.502569 27666 raft_consensus.cc:729] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.502646 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 19295fe971e7409384b39937ea5d80e3 (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.502672 27666 consensus_queue.cc:260] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.502848 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19295fe971e7409384b39937ea5d80e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.502923 27666 ts_tablet_manager.cc:1398] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:35.502904 26774 tablet_service.cc:1467] Processing CreateTablet for tablet c618121cd88d4a53a6badd33b99c439c (DEFAULT_TABLE table=rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.503073 27666 tablet_bootstrap.cc:492] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.503091 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c618121cd88d4a53a6badd33b99c439c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.503841 27666 tablet_bootstrap.cc:654] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.504724 27665 tablet_bootstrap.cc:492] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:35.504801 27665 ts_tablet_manager.cc:1367] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:35.505026 27664 tablet_bootstrap.cc:492] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.505112 27664 ts_tablet_manager.cc:1367] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:35.505148 27665 raft_consensus.cc:348] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.505242 27665 raft_consensus.cc:374] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.505277 27665 raft_consensus.cc:729] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.505417 27665 consensus_queue.cc:260] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.505518 27664 raft_consensus.cc:348] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.505612 27664 raft_consensus.cc:374] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.505643 27664 raft_consensus.cc:729] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.505658 27665 ts_tablet_manager.cc:1398] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:35.505749 27664 consensus_queue.cc:260] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.505807 27665 tablet_bootstrap.cc:492] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:35.505869 27663 tablet_bootstrap.cc:492] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.505934 27663 ts_tablet_manager.cc:1367] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:58:35.506016 27664 ts_tablet_manager.cc:1398] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.506347 27663 raft_consensus.cc:348] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.506440 27663 raft_consensus.cc:374] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.506469 27663 raft_consensus.cc:729] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.506567 27663 consensus_queue.cc:260] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.506810 27663 ts_tablet_manager.cc:1398] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:35.506819 27665 tablet_bootstrap.cc:654] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.506971 27663 tablet_bootstrap.cc:492] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.508054 27663 tablet_bootstrap.cc:654] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.508558 27666 tablet_bootstrap.cc:492] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.508628 27666 ts_tablet_manager.cc:1367] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 01:58:35.509001 27666 raft_consensus.cc:348] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.509088 27666 raft_consensus.cc:374] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.509116 27666 raft_consensus.cc:729] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.509214 27666 consensus_queue.cc:260] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.509250 27665 tablet_bootstrap.cc:492] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:35.509335 27665 ts_tablet_manager.cc:1367] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:35.509488 27666 ts_tablet_manager.cc:1398] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:35.509699 27665 raft_consensus.cc:348] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.509799 27665 raft_consensus.cc:374] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.509836 27665 raft_consensus.cc:729] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.509945 27665 consensus_queue.cc:260] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.510406 27663 tablet_bootstrap.cc:492] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.510491 27663 ts_tablet_manager.cc:1367] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 01:58:35.510990 27663 raft_consensus.cc:348] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.511199 27663 raft_consensus.cc:374] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.511248 27663 raft_consensus.cc:729] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.511346 27663 consensus_queue.cc:260] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.511618 27663 ts_tablet_manager.cc:1398] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:35.511757 27663 tablet_bootstrap.cc:492] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.511263 27665 ts_tablet_manager.cc:1398] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:35.511991 27665 tablet_bootstrap.cc:492] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:35.512554 27663 tablet_bootstrap.cc:654] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.512758 27665 tablet_bootstrap.cc:654] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.513639 27663 tablet_bootstrap.cc:492] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.513710 27663 ts_tablet_manager.cc:1367] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.514003 27665 tablet_bootstrap.cc:492] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:35.514102 27665 ts_tablet_manager.cc:1367] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:35.514109 27663 raft_consensus.cc:348] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.514199 27663 raft_consensus.cc:374] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.514225 27663 raft_consensus.cc:729] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.514343 27663 consensus_queue.cc:260] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.514499 27665 raft_consensus.cc:348] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.514575 27663 ts_tablet_manager.cc:1398] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.514585 27665 raft_consensus.cc:374] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.514616 27665 raft_consensus.cc:729] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.514725 27665 consensus_queue.cc:260] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.514999 27665 ts_tablet_manager.cc:1398] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:35.539523 27671 raft_consensus.cc:482] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:35.539595 27671 raft_consensus.cc:504] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.539992 27671 leader_election.cc:288] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:35.540233 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19295fe971e7409384b39937ea5d80e3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:35.540336 26930 raft_consensus.cc:2454] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:35.540593 26998 leader_election.cc:302] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:35.540757 27671 raft_consensus.cc:2781] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:35.540808 27671 raft_consensus.cc:482] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:35.540834 27671 raft_consensus.cc:3037] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.541630 27671 raft_consensus.cc:504] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.541965 27671 leader_election.cc:288] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:35.542263 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19295fe971e7409384b39937ea5d80e3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:35.542354 26930 raft_consensus.cc:3037] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.543112 26930 raft_consensus.cc:2454] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:35.543377 26998 leader_election.cc:302] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:35.543553 27671 raft_consensus.cc:2781] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:35.543635 27671 raft_consensus.cc:686] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:35.543620 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19295fe971e7409384b39937ea5d80e3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:35.543673 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19295fe971e7409384b39937ea5d80e3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:35.543712 26795 raft_consensus.cc:2454] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:35.543751 27671 consensus_queue.cc:237] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.545111 26513 catalog_manager.cc:5484] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:35.606283 27675 raft_consensus.cc:482] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:35.606384 27675 raft_consensus.cc:504] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.606679 27675 leader_election.cc:288] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:35.606981 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42923ea9b5f438a8760fcff8b93e288" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:35.606992 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42923ea9b5f438a8760fcff8b93e288" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:35.607096 26661 raft_consensus.cc:2454] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:35.607100 27063 raft_consensus.cc:2454] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:35.607367 26729 leader_election.cc:302] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:35.607534 27675 raft_consensus.cc:2781] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:35.607587 27675 raft_consensus.cc:482] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:35.607612 27675 raft_consensus.cc:3037] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.608291 27675 raft_consensus.cc:504] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.608570 27675 leader_election.cc:288] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:35.608882 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42923ea9b5f438a8760fcff8b93e288" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:35.608892 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42923ea9b5f438a8760fcff8b93e288" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:35.608966 27063 raft_consensus.cc:3037] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.608990 26661 raft_consensus.cc:3037] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.609642 26661 raft_consensus.cc:2454] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:35.609661 27063 raft_consensus.cc:2454] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:35.609895 26729 leader_election.cc:302] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:35.610080 27675 raft_consensus.cc:2781] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:35.610164 27675 raft_consensus.cc:686] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:35.610291 27675 consensus_queue.cc:237] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:35.611644 26513 catalog_manager.cc:5484] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:35.719332 27763 raft_consensus.cc:482] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:35.719447 27763 raft_consensus.cc:504] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.719884 27763 leader_election.cc:288] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:35.720263 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c618121cd88d4a53a6badd33b99c439c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:35.720327 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c618121cd88d4a53a6badd33b99c439c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:35.720404 26794 raft_consensus.cc:2454] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:35.720466 26661 raft_consensus.cc:2454] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:35.720678 26997 leader_election.cc:302] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:35.720862 27763 raft_consensus.cc:2781] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:35.720927 27763 raft_consensus.cc:482] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:35.720955 27763 raft_consensus.cc:3037] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.721766 27763 raft_consensus.cc:504] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.722081 27763 leader_election.cc:288] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:35.722397 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c618121cd88d4a53a6badd33b99c439c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:35.722455 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c618121cd88d4a53a6badd33b99c439c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:35.722502 26794 raft_consensus.cc:3037] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.722564 26661 raft_consensus.cc:3037] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.723170 26794 raft_consensus.cc:2454] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:35.723230 26661 raft_consensus.cc:2454] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:35.723454 26997 leader_election.cc:302] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:35.723608 27763 raft_consensus.cc:2781] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:35.723709 27763 raft_consensus.cc:686] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:35.723852 27763 consensus_queue.cc:237] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:35.725244 26513 catalog_manager.cc:5484] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
W20240502 01:58:35.831182 27820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:35.831387 27820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:35.834260 27820 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 01:58:35.904366 26661 raft_consensus.cc:1261] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.904914 27063 raft_consensus.cc:1261] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.905550 26794 raft_consensus.cc:3037] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:35.905599 26795 raft_consensus.cc:1261] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.906044 27763 consensus_queue.cc:1035] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.906251 27683 consensus_queue.cc:1035] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.906476 26930 raft_consensus.cc:1261] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.906978 27671 consensus_queue.cc:1035] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.908808 27683 consensus_queue.cc:1035] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.909629 26661 raft_consensus.cc:1261] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.910518 26794 raft_consensus.cc:1261] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:35.915123 27763 consensus_queue.cc:1035] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:35.915452 27763 consensus_queue.cc:1035] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 98.2043 ms
time per row: 0.0491022 ms
I20240502 01:58:35.969674 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:35.975153 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 936e77ced55c41d6bfe955d9a4271b80 (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.975448 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 2c150446ded240a2946d5afa10ecd1df (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.975445 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 936e77ced55c41d6bfe955d9a4271b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.975780 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c150446ded240a2946d5afa10ecd1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.975742 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 2e129aa347444eafab51845d9bf2ae32 (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.975983 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e129aa347444eafab51845d9bf2ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.977667 27663 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.978143 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 936e77ced55c41d6bfe955d9a4271b80 (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.978365 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 936e77ced55c41d6bfe955d9a4271b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.978515 27663 tablet_bootstrap.cc:654] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.979935 27664 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.980597 27664 tablet_bootstrap.cc:654] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.975749 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 2c150446ded240a2946d5afa10ecd1df (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.981496 27664 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.981516 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c150446ded240a2946d5afa10ecd1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.981564 27664 ts_tablet_manager.cc:1367] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:35.981557 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 936e77ced55c41d6bfe955d9a4271b80 (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.981688 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 2e129aa347444eafab51845d9bf2ae32 (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.981807 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 936e77ced55c41d6bfe955d9a4271b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.981978 27664 raft_consensus.cc:348] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.982108 27664 raft_consensus.cc:374] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.982141 27664 raft_consensus.cc:729] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.982110 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 2e129aa347444eafab51845d9bf2ae32 (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.982262 27664 consensus_queue.cc:260] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.982372 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e129aa347444eafab51845d9bf2ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.982515 27664 ts_tablet_manager.cc:1398] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:35.982679 27664 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.983489 27664 tablet_bootstrap.cc:654] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.984308 27665 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:35.984455 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 2c150446ded240a2946d5afa10ecd1df (DEFAULT_TABLE table=rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:35.984643 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c150446ded240a2946d5afa10ecd1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.985086 27665 tablet_bootstrap.cc:654] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.986382 27665 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:35.986462 27665 ts_tablet_manager.cc:1367] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:35.986877 27665 raft_consensus.cc:348] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.986968 27665 raft_consensus.cc:374] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.986995 27665 raft_consensus.cc:729] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.987103 27665 consensus_queue.cc:260] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.987375 27665 ts_tablet_manager.cc:1398] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.988719 27663 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.988804 27663 ts_tablet_manager.cc:1367] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240502 01:58:35.988821 27664 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.988880 27664 ts_tablet_manager.cc:1367] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:35.989215 27663 raft_consensus.cc:348] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.989239 27664 raft_consensus.cc:348] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.989311 27664 raft_consensus.cc:374] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.989311 27663 raft_consensus.cc:374] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.989341 27664 raft_consensus.cc:729] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.989341 27663 raft_consensus.cc:729] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.989446 27664 consensus_queue.cc:260] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.989446 27663 consensus_queue.cc:260] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.989707 27664 ts_tablet_manager.cc:1398] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.989701 27663 ts_tablet_manager.cc:1398] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:35.989837 27663 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:35.989837 27664 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:35.990630 27666 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.990837 27663 tablet_bootstrap.cc:654] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.990978 27664 tablet_bootstrap.cc:654] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.991417 27666 tablet_bootstrap.cc:654] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.991909 27664 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:35.991974 27663 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:35.991976 27664 ts_tablet_manager.cc:1367] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:35.992030 27663 ts_tablet_manager.cc:1367] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.992352 27664 raft_consensus.cc:348] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.992388 27663 raft_consensus.cc:348] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.992439 27664 raft_consensus.cc:374] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.992462 27663 raft_consensus.cc:374] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.992467 27664 raft_consensus.cc:729] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.992491 27663 raft_consensus.cc:729] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.992573 27664 consensus_queue.cc:260] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.992573 27663 consensus_queue.cc:260] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.992810 27663 ts_tablet_manager.cc:1398] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:35.992830 27664 ts_tablet_manager.cc:1398] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:35.993446 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e129aa347444eafab51845d9bf2ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:35.994978 27666 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.995051 27666 ts_tablet_manager.cc:1367] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:35.995450 27666 raft_consensus.cc:348] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.995550 27666 raft_consensus.cc:374] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.995576 27666 raft_consensus.cc:729] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.995682 27666 consensus_queue.cc:260] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:35.995982 27666 ts_tablet_manager.cc:1398] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:35.996140 27666 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:35.996914 27665 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:35.996934 27666 tablet_bootstrap.cc:654] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.997678 27665 tablet_bootstrap.cc:654] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:35.999011 27666 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:35.999090 27666 ts_tablet_manager.cc:1367] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:58:35.999260 27665 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:35.999339 27665 ts_tablet_manager.cc:1367] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:35.999472 27666 raft_consensus.cc:348] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.999564 27666 raft_consensus.cc:374] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.999593 27666 raft_consensus.cc:729] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.999709 27666 consensus_queue.cc:260] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.999708 27665 raft_consensus.cc:348] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.999788 27665 raft_consensus.cc:374] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:35.999810 27665 raft_consensus.cc:729] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:35.999912 27665 consensus_queue.cc:260] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:35.999951 27666 ts_tablet_manager.cc:1398] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:36.000164 27665 ts_tablet_manager.cc:1398] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:36.167224 27683 raft_consensus.cc:482] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:36.167331 27683 raft_consensus.cc:504] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.167742 27683 leader_election.cc:288] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:36.168113 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e129aa347444eafab51845d9bf2ae32" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:36.168247 26661 raft_consensus.cc:2454] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:36.168540 26729 leader_election.cc:302] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.168705 27683 raft_consensus.cc:2781] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:36.168749 27683 raft_consensus.cc:482] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:36.168784 27683 raft_consensus.cc:3037] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.169606 27683 raft_consensus.cc:504] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.169974 27683 leader_election.cc:288] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:36.170333 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e129aa347444eafab51845d9bf2ae32" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:36.170447 26661 raft_consensus.cc:3037] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.171379 26661 raft_consensus.cc:2454] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:36.171645 26729 leader_election.cc:302] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.171700 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e129aa347444eafab51845d9bf2ae32" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:36.171756 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e129aa347444eafab51845d9bf2ae32" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:36.171829 26930 raft_consensus.cc:2454] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:36.171891 27683 raft_consensus.cc:2781] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:36.171978 27683 raft_consensus.cc:686] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:36.172111 27683 consensus_queue.cc:237] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.173549 26510 catalog_manager.cc:5484] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:36.202419 27672 raft_consensus.cc:482] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:36.202503 27672 raft_consensus.cc:504] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.202791 27672 leader_election.cc:288] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:36.203114 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c150446ded240a2946d5afa10ecd1df" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:36.203137 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c150446ded240a2946d5afa10ecd1df" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:36.203238 27063 raft_consensus.cc:2454] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:36.203239 26795 raft_consensus.cc:2454] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:36.203533 26861 leader_election.cc:302] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.203683 27672 raft_consensus.cc:2781] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:36.203742 27672 raft_consensus.cc:482] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:36.203769 27672 raft_consensus.cc:3037] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.204425 27672 raft_consensus.cc:504] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.204720 27672 leader_election.cc:288] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:36.204998 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c150446ded240a2946d5afa10ecd1df" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:36.205051 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c150446ded240a2946d5afa10ecd1df" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:36.205091 27063 raft_consensus.cc:3037] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.205147 26795 raft_consensus.cc:3037] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.205747 27063 raft_consensus.cc:2454] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:36.205811 26795 raft_consensus.cc:2454] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:36.206005 26862 leader_election.cc:302] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:36.206185 27672 raft_consensus.cc:2781] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:36.206315 27672 raft_consensus.cc:686] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:36.206435 27672 consensus_queue.cc:237] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.207841 26510 catalog_manager.cc:5484] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:36.311575 27849 raft_consensus.cc:482] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:36.311667 27849 raft_consensus.cc:504] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.312007 27849 leader_election.cc:288] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:36.312322 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "936e77ced55c41d6bfe955d9a4271b80" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:36.312355 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "936e77ced55c41d6bfe955d9a4271b80" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:36.312439 26661 raft_consensus.cc:2454] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:36.312479 26929 raft_consensus.cc:2454] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:36.312686 26997 leader_election.cc:302] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:36.312856 27849 raft_consensus.cc:2781] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:36.312914 27849 raft_consensus.cc:482] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:36.312939 27849 raft_consensus.cc:3037] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.313685 27849 raft_consensus.cc:504] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.313985 27849 leader_election.cc:288] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:36.314272 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "936e77ced55c41d6bfe955d9a4271b80" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:36.314282 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "936e77ced55c41d6bfe955d9a4271b80" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:36.314373 26929 raft_consensus.cc:3037] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.314401 26661 raft_consensus.cc:3037] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.315059 26661 raft_consensus.cc:2454] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:36.315127 26929 raft_consensus.cc:2454] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:36.315331 26997 leader_election.cc:302] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:36.315486 27849 raft_consensus.cc:2781] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:36.315573 27849 raft_consensus.cc:686] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:36.315704 27849 consensus_queue.cc:237] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.317201 26510 catalog_manager.cc:5484] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:36.456941 27873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:36.457123 27873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:36.460140 27873 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 01:58:36.541829 26795 raft_consensus.cc:1261] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:36.542469 27672 consensus_queue.cc:1035] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:36.544263 26929 raft_consensus.cc:3037] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.544313 26661 raft_consensus.cc:1261] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:36.544749 27683 consensus_queue.cc:1035] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:36.545342 26929 raft_consensus.cc:1261] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:36.546391 27683 consensus_queue.cc:1035] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:36.550876 27063 raft_consensus.cc:1261] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:36.551283 26660 raft_consensus.cc:1261] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:36.551653 27870 consensus_queue.cc:1035] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:36.551815 26930 raft_consensus.cc:1261] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:36.552628 27849 consensus_queue.cc:1035] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:36.553939 27849 consensus_queue.cc:1035] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 120.064 ms
time per row: 0.060032 ms
I20240502 01:58:36.616851 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:36.621562 26640 tablet_service.cc:1467] Processing CreateTablet for tablet ce3d632b976e491e8ac726952c69cbc3 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.621621 26775 tablet_service.cc:1467] Processing CreateTablet for tablet ce3d632b976e491e8ac726952c69cbc3 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.621851 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3d632b976e491e8ac726952c69cbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.621866 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3d632b976e491e8ac726952c69cbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.622377 26774 tablet_service.cc:1467] Processing CreateTablet for tablet c0d1a201768f4f68b3bca4197d6d15c2 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.622494 26907 tablet_service.cc:1467] Processing CreateTablet for tablet ce3d632b976e491e8ac726952c69cbc3 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.622588 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d1a201768f4f68b3bca4197d6d15c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.622700 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3d632b976e491e8ac726952c69cbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.622920 27043 tablet_service.cc:1467] Processing CreateTablet for tablet e812884ef6474b5384d026637dc62b61 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.623144 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e812884ef6474b5384d026637dc62b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.624135 27044 tablet_service.cc:1467] Processing CreateTablet for tablet c0d1a201768f4f68b3bca4197d6d15c2 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.624326 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d1a201768f4f68b3bca4197d6d15c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.625173 27935 tablet_bootstrap.cc:492] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:36.625437 26908 tablet_service.cc:1467] Processing CreateTablet for tablet e812884ef6474b5384d026637dc62b61 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.625641 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e812884ef6474b5384d026637dc62b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.626189 27935 tablet_bootstrap.cc:654] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.626669 26640 tablet_service.cc:1467] Processing CreateTablet for tablet c0d1a201768f4f68b3bca4197d6d15c2 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.626880 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d1a201768f4f68b3bca4197d6d15c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.627055 27937 tablet_bootstrap.cc:492] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:36.627624 26639 tablet_service.cc:1467] Processing CreateTablet for tablet e812884ef6474b5384d026637dc62b61 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:36.627825 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e812884ef6474b5384d026637dc62b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:36.627848 27937 tablet_bootstrap.cc:654] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.627902 27934 tablet_bootstrap.cc:492] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:36.628665 27934 tablet_bootstrap.cc:654] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.628785 27933 tablet_bootstrap.cc:492] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:36.628821 27937 tablet_bootstrap.cc:492] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:36.628890 27937 ts_tablet_manager.cc:1367] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:36.629297 27937 raft_consensus.cc:348] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.629386 27937 raft_consensus.cc:374] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.629415 27937 raft_consensus.cc:729] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.629575 27933 tablet_bootstrap.cc:654] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.629529 27937 consensus_queue.cc:260] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.629792 27937 ts_tablet_manager.cc:1398] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:36.629967 27937 tablet_bootstrap.cc:492] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:36.630613 27933 tablet_bootstrap.cc:492] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:36.630694 27933 ts_tablet_manager.cc:1367] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:36.630764 27937 tablet_bootstrap.cc:654] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.630842 27935 tablet_bootstrap.cc:492] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:36.630911 27935 ts_tablet_manager.cc:1367] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:36.631083 27933 raft_consensus.cc:348] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.631171 27933 raft_consensus.cc:374] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.631201 27933 raft_consensus.cc:729] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.631276 27935 raft_consensus.cc:348] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.631315 27933 consensus_queue.cc:260] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.631369 27935 raft_consensus.cc:374] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.631400 27935 raft_consensus.cc:729] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.631505 27935 consensus_queue.cc:260] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.631575 27933 ts_tablet_manager.cc:1398] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:36.631671 27937 tablet_bootstrap.cc:492] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:36.631740 27933 tablet_bootstrap.cc:492] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:36.631742 27937 ts_tablet_manager.cc:1367] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:36.631747 27935 ts_tablet_manager.cc:1398] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:36.631892 27935 tablet_bootstrap.cc:492] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:36.632122 27937 raft_consensus.cc:348] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.632206 27937 raft_consensus.cc:374] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.632233 27937 raft_consensus.cc:729] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.632318 27937 consensus_queue.cc:260] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.632529 27933 tablet_bootstrap.cc:654] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.632550 27937 ts_tablet_manager.cc:1398] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:36.632568 27935 tablet_bootstrap.cc:654] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.633899 27934 tablet_bootstrap.cc:492] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:36.634078 27934 ts_tablet_manager.cc:1367] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:36.634919 27934 raft_consensus.cc:348] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.635056 27934 raft_consensus.cc:374] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.635102 27934 raft_consensus.cc:729] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.635129 27933 tablet_bootstrap.cc:492] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:36.635210 27933 ts_tablet_manager.cc:1367] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:36.635213 27934 consensus_queue.cc:260] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.635474 27934 ts_tablet_manager.cc:1398] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:36.635588 27933 raft_consensus.cc:348] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.635650 27934 tablet_bootstrap.cc:492] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:36.635675 27933 raft_consensus.cc:374] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.635710 27933 raft_consensus.cc:729] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.635815 27933 consensus_queue.cc:260] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.636353 27934 tablet_bootstrap.cc:654] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.637313 27934 tablet_bootstrap.cc:492] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:36.637405 27935 tablet_bootstrap.cc:492] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:36.637418 27934 ts_tablet_manager.cc:1367] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:36.637490 27935 ts_tablet_manager.cc:1367] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:36.637560 27933 ts_tablet_manager.cc:1398] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:36.637714 27933 tablet_bootstrap.cc:492] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:36.637820 27935 raft_consensus.cc:348] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.637918 27935 raft_consensus.cc:374] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.637959 27935 raft_consensus.cc:729] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.637979 27934 raft_consensus.cc:348] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.638154 27934 raft_consensus.cc:374] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.638103 27935 consensus_queue.cc:260] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.638203 27934 raft_consensus.cc:729] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.638378 27934 consensus_queue.cc:260] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.638482 27935 ts_tablet_manager.cc:1398] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:36.638562 27933 tablet_bootstrap.cc:654] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:36.638746 27934 ts_tablet_manager.cc:1398] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:36.639655 27933 tablet_bootstrap.cc:492] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:36.639768 27933 ts_tablet_manager.cc:1367] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:36.640161 27933 raft_consensus.cc:348] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.640239 27933 raft_consensus.cc:374] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:36.640266 27933 raft_consensus.cc:729] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:36.640381 27933 consensus_queue.cc:260] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.640633 27933 ts_tablet_manager.cc:1398] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:36.707316 27848 raft_consensus.cc:482] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:36.707406 27848 raft_consensus.cc:504] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.707736 27848 leader_election.cc:288] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:36.708082 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e812884ef6474b5384d026637dc62b61" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:36.708083 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e812884ef6474b5384d026637dc62b61" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:36.708214 26661 raft_consensus.cc:2454] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:36.708214 26929 raft_consensus.cc:2454] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:36.708467 26998 leader_election.cc:302] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:36.708611 27848 raft_consensus.cc:2781] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:36.708669 27848 raft_consensus.cc:482] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:36.708698 27848 raft_consensus.cc:3037] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.709445 27848 raft_consensus.cc:504] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.709729 27848 leader_election.cc:288] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:36.710002 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e812884ef6474b5384d026637dc62b61" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:36.710093 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e812884ef6474b5384d026637dc62b61" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:36.710143 26661 raft_consensus.cc:3037] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.710193 26929 raft_consensus.cc:3037] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.710987 26661 raft_consensus.cc:2454] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:36.710987 26929 raft_consensus.cc:2454] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:36.711246 26998 leader_election.cc:302] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:36.711390 27848 raft_consensus.cc:2781] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:36.711475 27848 raft_consensus.cc:686] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:36.711599 27848 consensus_queue.cc:237] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.712997 26512 catalog_manager.cc:5484] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:36.764997 27675 raft_consensus.cc:482] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:36.765085 27675 raft_consensus.cc:504] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.765434 27675 leader_election.cc:288] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:36.765723 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d1a201768f4f68b3bca4197d6d15c2" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:36.765766 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d1a201768f4f68b3bca4197d6d15c2" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:36.765851 26661 raft_consensus.cc:2454] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:36.765883 27063 raft_consensus.cc:2454] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:36.766112 26729 leader_election.cc:302] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.766264 27675 raft_consensus.cc:2781] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:36.766317 27675 raft_consensus.cc:482] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:36.766363 27675 raft_consensus.cc:3037] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.767069 27675 raft_consensus.cc:504] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.767378 27675 leader_election.cc:288] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:36.767633 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d1a201768f4f68b3bca4197d6d15c2" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:36.767673 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d1a201768f4f68b3bca4197d6d15c2" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:36.767730 27063 raft_consensus.cc:3037] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.767753 26661 raft_consensus.cc:3037] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.768371 27063 raft_consensus.cc:2454] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:36.768373 26661 raft_consensus.cc:2454] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:36.768612 26729 leader_election.cc:302] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.768762 27675 raft_consensus.cc:2781] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:36.768842 27675 raft_consensus.cc:686] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:36.768954 27675 consensus_queue.cc:237] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:36.770409 26512 catalog_manager.cc:5484] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:36.771020 27675 raft_consensus.cc:482] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:36.771092 27675 raft_consensus.cc:504] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.771373 27675 leader_election.cc:288] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:36.771651 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3d632b976e491e8ac726952c69cbc3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:36.771755 26661 raft_consensus.cc:2454] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:36.771762 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3d632b976e491e8ac726952c69cbc3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:36.771867 26929 raft_consensus.cc:2454] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:36.772044 26729 leader_election.cc:302] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.772315 27675 raft_consensus.cc:2781] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:36.772372 27675 raft_consensus.cc:482] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:36.772409 27675 raft_consensus.cc:3037] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.773008 27675 raft_consensus.cc:504] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.773288 27675 leader_election.cc:288] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:36.773561 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3d632b976e491e8ac726952c69cbc3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:36.773561 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3d632b976e491e8ac726952c69cbc3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:36.773661 26661 raft_consensus.cc:3037] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.773661 26929 raft_consensus.cc:3037] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:36.774502 26929 raft_consensus.cc:2454] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:36.774502 26661 raft_consensus.cc:2454] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:36.774745 26729 leader_election.cc:302] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:36.774981 27675 raft_consensus.cc:2781] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:36.775067 27675 raft_consensus.cc:686] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:36.775184 27675 consensus_queue.cc:237] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:36.776479 26512 catalog_manager.cc:5484] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:36.912904 27950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:36.913093 27950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:36.916062 27950 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 01:58:36.999322 26661 raft_consensus.cc:1261] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:36.999661 27063 raft_consensus.cc:1261] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.000002 26929 raft_consensus.cc:1261] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.000715 27683 consensus_queue.cc:1035] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.000908 27675 consensus_queue.cc:1035] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.001055 27683 consensus_queue.cc:1035] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.007400 26930 raft_consensus.cc:1261] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.007829 27849 consensus_queue.cc:1035] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.008955 26661 raft_consensus.cc:1261] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.008987 26660 raft_consensus.cc:1261] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.009490 27683 consensus_queue.cc:1035] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.009624 27848 consensus_queue.cc:1035] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.032233 27911 mvcc.cc:205] Tried to move back new op lower bound from 7023063519239499776 to 7023063518051762176. Current Snapshot: MvccSnapshot[applied={T|T < 7023063519239499776}]
I20240502 01:58:37.046114 27912 mvcc.cc:205] Tried to move back new op lower bound from 7023063519280287744 to 7023063518051762176. Current Snapshot: MvccSnapshot[applied={T|T < 7023063519239499776}]
INSERT report
rows total: 2000
time total: 105.599 ms
time per row: 0.0527996 ms
I20240502 01:58:37.059067 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:37.063997 26908 tablet_service.cc:1467] Processing CreateTablet for tablet cceb8a91997e4db9a4842df26ad3b02e (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.064128 26775 tablet_service.cc:1467] Processing CreateTablet for tablet cceb8a91997e4db9a4842df26ad3b02e (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.064280 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cceb8a91997e4db9a4842df26ad3b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.064385 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cceb8a91997e4db9a4842df26ad3b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.064946 26909 tablet_service.cc:1467] Processing CreateTablet for tablet dcf3a090ea284d12979b161f59081ff7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.065153 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf3a090ea284d12979b161f59081ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.065397 27044 tablet_service.cc:1467] Processing CreateTablet for tablet cceb8a91997e4db9a4842df26ad3b02e (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.065614 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cceb8a91997e4db9a4842df26ad3b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.066529 27937 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:37.066612 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 565aab51035c4f328e49eca53eddf621 (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.066823 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565aab51035c4f328e49eca53eddf621. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.067320 27937 tablet_bootstrap.cc:654] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.067611 27935 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:37.067780 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 565aab51035c4f328e49eca53eddf621 (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.067978 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565aab51035c4f328e49eca53eddf621. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.068348 27935 tablet_bootstrap.cc:654] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.068415 27934 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:37.069137 27934 tablet_bootstrap.cc:654] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.069324 27935 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:37.069396 27935 ts_tablet_manager.cc:1367] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:37.069792 27935 raft_consensus.cc:348] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.069886 27935 raft_consensus.cc:374] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.069924 27935 raft_consensus.cc:729] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.070046 27935 consensus_queue.cc:260] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.070087 26774 tablet_service.cc:1467] Processing CreateTablet for tablet dcf3a090ea284d12979b161f59081ff7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.070156 26639 tablet_service.cc:1467] Processing CreateTablet for tablet dcf3a090ea284d12979b161f59081ff7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.070308 27935 ts_tablet_manager.cc:1398] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.070294 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf3a090ea284d12979b161f59081ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.070292 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 565aab51035c4f328e49eca53eddf621 (DEFAULT_TABLE table=rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.070398 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf3a090ea284d12979b161f59081ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.070462 27935 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:37.070518 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565aab51035c4f328e49eca53eddf621. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.071233 27935 tablet_bootstrap.cc:654] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.072605 27933 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:37.073406 27933 tablet_bootstrap.cc:654] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.073943 27937 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:37.073951 27935 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:37.074052 27937 ts_tablet_manager.cc:1367] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:58:37.074072 27935 ts_tablet_manager.cc:1367] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:37.074433 27935 raft_consensus.cc:348] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.074469 27937 raft_consensus.cc:348] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.074532 27934 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:37.074533 27935 raft_consensus.cc:374] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.074559 27935 raft_consensus.cc:729] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.074559 27937 raft_consensus.cc:374] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.074587 27937 raft_consensus.cc:729] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.074599 27934 ts_tablet_manager.cc:1367] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 01:58:37.074648 27935 consensus_queue.cc:260] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.074700 27937 consensus_queue.cc:260] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.074877 27935 ts_tablet_manager.cc:1398] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.074950 27937 ts_tablet_manager.cc:1398] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:37.074990 27934 raft_consensus.cc:348] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.075070 27934 raft_consensus.cc:374] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.075099 27934 raft_consensus.cc:729] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.075116 27937 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:37.075219 27934 consensus_queue.cc:260] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.075460 27934 ts_tablet_manager.cc:1398] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:58:37.075600 27934 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:37.075875 27937 tablet_bootstrap.cc:654] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.076361 27934 tablet_bootstrap.cc:654] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.077538 27937 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:37.077611 27937 ts_tablet_manager.cc:1367] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:58:37.077819 27934 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:37.077884 27934 ts_tablet_manager.cc:1367] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:37.077982 27937 raft_consensus.cc:348] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.078171 27937 raft_consensus.cc:374] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.078208 27937 raft_consensus.cc:729] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.078284 27934 raft_consensus.cc:348] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.078321 27937 consensus_queue.cc:260] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.078361 27934 raft_consensus.cc:374] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.078387 27934 raft_consensus.cc:729] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.078488 27934 consensus_queue.cc:260] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.078786 27934 ts_tablet_manager.cc:1398] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.078804 27937 ts_tablet_manager.cc:1398] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:37.078948 27937 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:37.078948 27934 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:37.079758 27934 tablet_bootstrap.cc:654] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.079874 27937 tablet_bootstrap.cc:654] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.080732 27934 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:37.080811 27934 ts_tablet_manager.cc:1367] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:37.081120 27934 raft_consensus.cc:348] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.081194 27934 raft_consensus.cc:374] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.081221 27934 raft_consensus.cc:729] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.081355 27934 consensus_queue.cc:260] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.081593 27934 ts_tablet_manager.cc:1398] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.082019 27933 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:37.082129 27933 ts_tablet_manager.cc:1367] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 01:58:37.082468 27933 raft_consensus.cc:348] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.082552 27933 raft_consensus.cc:374] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.082580 27933 raft_consensus.cc:729] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.082698 27933 consensus_queue.cc:260] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.082952 27933 ts_tablet_manager.cc:1398] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:37.084105 27937 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:37.084177 27937 ts_tablet_manager.cc:1367] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 01:58:37.084568 27937 raft_consensus.cc:348] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.084654 27937 raft_consensus.cc:374] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.084681 27937 raft_consensus.cc:729] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.084784 27937 consensus_queue.cc:260] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.085103 27937 ts_tablet_manager.cc:1398] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.105513 27977 raft_consensus.cc:482] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:37.105607 27977 raft_consensus.cc:504] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.105893 27977 leader_election.cc:288] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:37.106194 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf3a090ea284d12979b161f59081ff7" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:37.106215 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf3a090ea284d12979b161f59081ff7" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:37.106307 26929 raft_consensus.cc:2454] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:37.106320 26661 raft_consensus.cc:2454] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:37.106531 26730 leader_election.cc:302] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.106693 27977 raft_consensus.cc:2781] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:37.106753 27977 raft_consensus.cc:482] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:37.106796 27977 raft_consensus.cc:3037] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.107425 27977 raft_consensus.cc:504] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.107710 27977 leader_election.cc:288] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:37.107937 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf3a090ea284d12979b161f59081ff7" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:37.107975 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcf3a090ea284d12979b161f59081ff7" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:37.108021 26929 raft_consensus.cc:3037] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.108090 26661 raft_consensus.cc:3037] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.108644 26929 raft_consensus.cc:2454] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:37.108744 26661 raft_consensus.cc:2454] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:37.108965 26730 leader_election.cc:302] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.109128 27977 raft_consensus.cc:2781] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:37.109216 27977 raft_consensus.cc:686] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:37.109328 27977 consensus_queue.cc:237] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:37.109800 27985 raft_consensus.cc:482] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:37.109891 27985 raft_consensus.cc:504] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.110284 27985 leader_election.cc:288] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:37.110512 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cceb8a91997e4db9a4842df26ad3b02e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:37.110539 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cceb8a91997e4db9a4842df26ad3b02e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:37.110607 26929 raft_consensus.cc:2454] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:37.110620 26795 raft_consensus.cc:2454] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:37.110868 26997 leader_election.cc:302] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.110859 26509 catalog_manager.cc:5484] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:37.111066 27985 raft_consensus.cc:2781] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:37.111117 27985 raft_consensus.cc:482] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:37.111158 27985 raft_consensus.cc:3037] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.111840 27985 raft_consensus.cc:504] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.112160 27985 leader_election.cc:288] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:37.112408 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cceb8a91997e4db9a4842df26ad3b02e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:37.112502 26929 raft_consensus.cc:3037] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.112488 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cceb8a91997e4db9a4842df26ad3b02e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:37.112562 26795 raft_consensus.cc:3037] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.113195 26929 raft_consensus.cc:2454] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:37.113451 26998 leader_election.cc:302] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:37.113615 27985 raft_consensus.cc:2781] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:37.113710 27985 raft_consensus.cc:686] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:37.113826 27985 consensus_queue.cc:237] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.115119 26509 catalog_manager.cc:5484] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:37.130614 26795 raft_consensus.cc:2454] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:37.153703 27977 raft_consensus.cc:482] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:37.153780 27977 raft_consensus.cc:504] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.154093 27977 leader_election.cc:288] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:37.154359 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565aab51035c4f328e49eca53eddf621" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:37.154439 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565aab51035c4f328e49eca53eddf621" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:37.154556 27063 raft_consensus.cc:2454] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:37.154748 26929 raft_consensus.cc:2454] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:37.154786 26730 leader_election.cc:302] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.154934 27977 raft_consensus.cc:2781] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:37.154995 27977 raft_consensus.cc:482] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:37.155035 27977 raft_consensus.cc:3037] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.155642 27977 raft_consensus.cc:504] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.155915 27977 leader_election.cc:288] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:37.156168 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565aab51035c4f328e49eca53eddf621" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:37.156263 26929 raft_consensus.cc:3037] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.156250 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565aab51035c4f328e49eca53eddf621" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:37.156355 27064 raft_consensus.cc:3037] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.156872 26929 raft_consensus.cc:2454] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:37.156987 27064 raft_consensus.cc:2454] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:37.157095 26730 leader_election.cc:302] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.157259 27977 raft_consensus.cc:2781] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:37.157334 27977 raft_consensus.cc:686] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:37.157474 27977 consensus_queue.cc:237] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:37.158774 26509 catalog_manager.cc:5484] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
W20240502 01:58:37.265095 27999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:37.265283 27999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:37.268265 27999 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 01:58:37.349629 26929 raft_consensus.cc:1261] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.349623 26795 raft_consensus.cc:1261] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.350056 27848 consensus_queue.cc:1035] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.350277 27985 consensus_queue.cc:1035] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.352537 26661 raft_consensus.cc:1261] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.353063 27977 consensus_queue.cc:1035] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.353363 27064 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.353668 26930 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.356809 27675 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.357079 27675 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.357542 28023 mvcc.cc:205] Tried to move back new op lower bound from 7023063520660234240 to 7023063519699517440. Current Snapshot: MvccSnapshot[applied={T|T < 7023063520660234240}]
I20240502 01:58:37.369652 26929 raft_consensus.cc:1261] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.369997 27683 consensus_queue.cc:1035] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.401263 27928 mvcc.cc:205] Tried to move back new op lower bound from 7023063520670982144 to 7023063519878070272. Current Snapshot: MvccSnapshot[applied={T|T < 7023063520670982144}]
INSERT report
rows total: 2000
time total: 110.738 ms
time per row: 0.0553688 ms
I20240502 01:58:37.417716 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:37.422864 26639 tablet_service.cc:1467] Processing CreateTablet for tablet b5439ff11ca04925bf46e736a5a53487 (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.422948 27043 tablet_service.cc:1467] Processing CreateTablet for tablet b5439ff11ca04925bf46e736a5a53487 (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.423179 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5439ff11ca04925bf46e736a5a53487. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.423206 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5439ff11ca04925bf46e736a5a53487. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.423519 26775 tablet_service.cc:1467] Processing CreateTablet for tablet b5439ff11ca04925bf46e736a5a53487 (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.423764 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5439ff11ca04925bf46e736a5a53487. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.424824 27935 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:37.425140 27933 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:37.425647 27935 tablet_bootstrap.cc:654] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.425908 27933 tablet_bootstrap.cc:654] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.426070 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 89a647fc0559437983278163148da132 (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.426146 27934 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:37.426306 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a647fc0559437983278163148da132. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.426402 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 89a647fc0559437983278163148da132 (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.426600 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a647fc0559437983278163148da132. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.426963 27934 tablet_bootstrap.cc:654] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.425658 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 1c002ce2d74547399685b69a0a4cbada (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.427755 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c002ce2d74547399685b69a0a4cbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.427932 27937 tablet_bootstrap.cc:492] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:37.428380 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 89a647fc0559437983278163148da132 (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.428575 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a647fc0559437983278163148da132. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.428656 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 1c002ce2d74547399685b69a0a4cbada (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.428740 27937 tablet_bootstrap.cc:654] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.428867 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c002ce2d74547399685b69a0a4cbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.428987 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 1c002ce2d74547399685b69a0a4cbada (DEFAULT_TABLE table=rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:37.429188 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c002ce2d74547399685b69a0a4cbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:37.430253 27934 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:37.430337 27934 ts_tablet_manager.cc:1367] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 01:58:37.430603 27937 tablet_bootstrap.cc:492] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:37.430684 27937 ts_tablet_manager.cc:1367] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:37.430747 27934 raft_consensus.cc:348] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.430866 27934 raft_consensus.cc:374] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.430895 27934 raft_consensus.cc:729] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.431000 27934 consensus_queue.cc:260] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.431077 27937 raft_consensus.cc:348] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.431160 27937 raft_consensus.cc:374] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.431188 27937 raft_consensus.cc:729] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.431298 27934 ts_tablet_manager.cc:1398] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:37.431293 27937 consensus_queue.cc:260] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.431459 27934 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:37.431545 27937 ts_tablet_manager.cc:1398] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.431669 27935 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:37.431751 27935 ts_tablet_manager.cc:1367] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:37.432335 27935 raft_consensus.cc:348] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.432432 27935 raft_consensus.cc:374] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.432443 27934 tablet_bootstrap.cc:654] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.432461 27935 raft_consensus.cc:729] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.432569 27935 consensus_queue.cc:260] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.432860 27935 ts_tablet_manager.cc:1398] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.433065 27935 tablet_bootstrap.cc:492] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:37.434418 27935 tablet_bootstrap.cc:654] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.434772 27934 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:37.434856 27934 ts_tablet_manager.cc:1367] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:58:37.435226 27934 raft_consensus.cc:348] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.435308 27934 raft_consensus.cc:374] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.435328 27934 raft_consensus.cc:729] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.435408 27935 tablet_bootstrap.cc:492] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:37.435431 27934 consensus_queue.cc:260] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.435484 27935 ts_tablet_manager.cc:1367] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:37.435710 27934 ts_tablet_manager.cc:1398] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:37.435855 27935 raft_consensus.cc:348] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.435950 27935 raft_consensus.cc:374] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.435981 27935 raft_consensus.cc:729] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.436074 27935 consensus_queue.cc:260] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.436317 27935 ts_tablet_manager.cc:1398] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:37.436479 27935 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:37.437297 27935 tablet_bootstrap.cc:654] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.438350 27935 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:37.438437 27935 ts_tablet_manager.cc:1367] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:37.438839 27935 raft_consensus.cc:348] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.438913 27935 raft_consensus.cc:374] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.438932 27935 raft_consensus.cc:729] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.439014 27935 consensus_queue.cc:260] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.439242 27935 ts_tablet_manager.cc:1398] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:37.446370 27933 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:37.446455 27933 ts_tablet_manager.cc:1367] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20240502 01:58:37.446887 27933 raft_consensus.cc:348] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.446995 27933 raft_consensus.cc:374] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.447026 27933 raft_consensus.cc:729] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.447142 27933 consensus_queue.cc:260] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.447428 27933 ts_tablet_manager.cc:1398] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:37.447727 27933 tablet_bootstrap.cc:492] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:37.448513 27933 tablet_bootstrap.cc:654] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.449524 27933 tablet_bootstrap.cc:492] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:37.449592 27933 ts_tablet_manager.cc:1367] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:37.449903 27933 raft_consensus.cc:348] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.449977 27933 raft_consensus.cc:374] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.450008 27933 raft_consensus.cc:729] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.450134 27933 consensus_queue.cc:260] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.450361 27933 ts_tablet_manager.cc:1398] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:37.450563 27933 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:37.451268 27933 tablet_bootstrap.cc:654] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:37.452126 27933 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:37.452203 27933 ts_tablet_manager.cc:1367] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:37.452602 27933 raft_consensus.cc:348] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.452697 27933 raft_consensus.cc:374] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:37.452734 27933 raft_consensus.cc:729] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:37.452827 27933 consensus_queue.cc:260] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.453047 27933 ts_tablet_manager.cc:1398] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:37.569504 27849 raft_consensus.cc:482] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:37.569610 27849 raft_consensus.cc:504] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.569944 27849 leader_election.cc:288] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:37.570376 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a647fc0559437983278163148da132" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:37.570376 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a647fc0559437983278163148da132" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:37.570535 26929 raft_consensus.cc:2454] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:37.570546 26661 raft_consensus.cc:2454] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:37.570855 26997 leader_election.cc:302] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:37.571015 27849 raft_consensus.cc:2781] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:37.571077 27849 raft_consensus.cc:482] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:37.571122 27849 raft_consensus.cc:3037] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.571880 27849 raft_consensus.cc:504] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.572218 27849 leader_election.cc:288] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:37.572487 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a647fc0559437983278163148da132" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:37.572541 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a647fc0559437983278163148da132" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:37.572598 26661 raft_consensus.cc:3037] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.572657 26929 raft_consensus.cc:3037] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.573343 26661 raft_consensus.cc:2454] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:37.573385 26929 raft_consensus.cc:2454] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:37.573612 26997 leader_election.cc:302] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:37.573793 27849 raft_consensus.cc:2781] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:37.573899 27849 raft_consensus.cc:686] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:37.574064 27849 consensus_queue.cc:237] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:37.575866 26510 catalog_manager.cc:5484] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:37.602844 27683 raft_consensus.cc:482] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:37.602929 27683 raft_consensus.cc:504] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.603231 27683 leader_election.cc:288] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:37.603518 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c002ce2d74547399685b69a0a4cbada" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:37.603528 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c002ce2d74547399685b69a0a4cbada" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:37.603618 26661 raft_consensus.cc:2454] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:37.603629 27064 raft_consensus.cc:2454] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:37.603909 26730 leader_election.cc:302] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.604051 27683 raft_consensus.cc:2781] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:37.604108 27683 raft_consensus.cc:482] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:37.604153 27683 raft_consensus.cc:3037] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.604848 27683 raft_consensus.cc:504] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.605132 27683 leader_election.cc:288] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:37.605430 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c002ce2d74547399685b69a0a4cbada" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:37.605430 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c002ce2d74547399685b69a0a4cbada" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:37.605540 27064 raft_consensus.cc:3037] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.605540 26661 raft_consensus.cc:3037] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.606422 27064 raft_consensus.cc:2454] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:37.606422 26661 raft_consensus.cc:2454] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:37.606683 26729 leader_election.cc:302] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:37.606849 27683 raft_consensus.cc:2781] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:37.606930 27683 raft_consensus.cc:686] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:37.607050 27683 consensus_queue.cc:237] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.608642 26514 catalog_manager.cc:5484] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:37.714277 27849 raft_consensus.cc:482] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:37.714404 27849 raft_consensus.cc:504] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.714804 27849 leader_election.cc:288] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:37.715246 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5439ff11ca04925bf46e736a5a53487" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:37.715284 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5439ff11ca04925bf46e736a5a53487" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:37.715410 26661 raft_consensus.cc:2454] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:37.715438 26795 raft_consensus.cc:2454] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:37.715730 26997 leader_election.cc:302] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:37.715955 27849 raft_consensus.cc:2781] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:37.716034 27849 raft_consensus.cc:482] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:37.716069 27849 raft_consensus.cc:3037] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.717062 27849 raft_consensus.cc:504] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.717366 27849 leader_election.cc:288] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:37.717751 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5439ff11ca04925bf46e736a5a53487" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:37.717756 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5439ff11ca04925bf46e736a5a53487" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:37.717850 26795 raft_consensus.cc:3037] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.717854 26661 raft_consensus.cc:3037] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:37.718550 26661 raft_consensus.cc:2454] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:37.718588 26795 raft_consensus.cc:2454] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:37.718815 26997 leader_election.cc:302] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:37.718976 27849 raft_consensus.cc:2781] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:37.719059 27849 raft_consensus.cc:686] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:37.719187 27849 consensus_queue.cc:237] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:37.720983 26514 catalog_manager.cc:5484] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:58:37.894917 28047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:37.895108 28047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:37.898083 28047 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 01:58:37.988481 26661 raft_consensus.cc:1261] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.989038 27683 consensus_queue.cc:1035] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.989383 27064 raft_consensus.cc:1261] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.991750 26795 raft_consensus.cc:1261] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.992132 27985 consensus_queue.cc:1035] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.993731 26930 raft_consensus.cc:1261] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.993861 27675 consensus_queue.cc:1035] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.994020 26659 raft_consensus.cc:1261] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.994671 27849 consensus_queue.cc:1035] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.995898 27985 consensus_queue.cc:1035] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:37.997138 26660 raft_consensus.cc:1261] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:37.997470 27985 consensus_queue.cc:1035] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 115.961 ms
time per row: 0.0579807 ms
I20240502 01:58:38.055862 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:38.060953 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 69df2cdb31d640ff9f43bb395f2f26fe (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.061249 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69df2cdb31d640ff9f43bb395f2f26fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.061342 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 47f84b6ffb074bc682aa7b087b8c3614 (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.061563 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f84b6ffb074bc682aa7b087b8c3614. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.061736 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 69df2cdb31d640ff9f43bb395f2f26fe (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.061990 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69df2cdb31d640ff9f43bb395f2f26fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.063509 28107 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:38.064332 28107 tablet_bootstrap.cc:654] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.064437 28108 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:38.064640 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 47f84b6ffb074bc682aa7b087b8c3614 (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.064836 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 69df2cdb31d640ff9f43bb395f2f26fe (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.064885 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f84b6ffb074bc682aa7b087b8c3614. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.065030 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69df2cdb31d640ff9f43bb395f2f26fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.065220 28108 tablet_bootstrap.cc:654] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.066298 28107 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:38.066370 28107 ts_tablet_manager.cc:1367] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:38.066874 28111 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:38.067145 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 47f84b6ffb074bc682aa7b087b8c3614 (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.067369 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f84b6ffb074bc682aa7b087b8c3614. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.067580 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 626774246b9445bc838427358e08f2cd (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.067710 28111 tablet_bootstrap.cc:654] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.067780 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626774246b9445bc838427358e08f2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.067816 28108 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:38.067881 28108 ts_tablet_manager.cc:1367] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:38.068296 28112 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:38.068286 28108 raft_consensus.cc:348] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.068387 28108 raft_consensus.cc:374] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.068418 28108 raft_consensus.cc:729] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.068513 28108 consensus_queue.cc:260] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.068751 28108 ts_tablet_manager.cc:1398] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:38.069101 28112 tablet_bootstrap.cc:654] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.069346 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 626774246b9445bc838427358e08f2cd (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.069561 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626774246b9445bc838427358e08f2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.070132 28111 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:38.070207 28111 ts_tablet_manager.cc:1367] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:38.070627 28111 raft_consensus.cc:348] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.070719 28111 raft_consensus.cc:374] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.070746 28111 raft_consensus.cc:729] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.066744 28107 raft_consensus.cc:348] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.070885 28107 raft_consensus.cc:374] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.070864 28111 consensus_queue.cc:260] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.070926 28107 raft_consensus.cc:729] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.071062 28107 consensus_queue.cc:260] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.071121 28111 ts_tablet_manager.cc:1398] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:38.071409 28107 ts_tablet_manager.cc:1398] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:38.071650 28111 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:38.071568 28107 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:38.072425 28107 tablet_bootstrap.cc:654] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.072463 28111 tablet_bootstrap.cc:654] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.073194 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 626774246b9445bc838427358e08f2cd (DEFAULT_TABLE table=rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.073390 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626774246b9445bc838427358e08f2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.073463 28112 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:38.073539 28112 ts_tablet_manager.cc:1367] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:38.073693 28111 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:38.073757 28111 ts_tablet_manager.cc:1367] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:38.073952 28112 raft_consensus.cc:348] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.074077 28112 raft_consensus.cc:374] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.074115 28112 raft_consensus.cc:729] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.074205 28112 consensus_queue.cc:260] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.074295 28111 raft_consensus.cc:348] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.074384 28111 raft_consensus.cc:374] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.074414 28111 raft_consensus.cc:729] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.074438 28112 ts_tablet_manager.cc:1398] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.074519 28111 consensus_queue.cc:260] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.074775 28111 ts_tablet_manager.cc:1398] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.075030 28112 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:38.075227 28108 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:38.076002 28108 tablet_bootstrap.cc:654] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.077003 28108 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:38.077076 28108 ts_tablet_manager.cc:1367] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:38.077344 28112 tablet_bootstrap.cc:654] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.077519 28108 raft_consensus.cc:348] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.077613 28108 raft_consensus.cc:374] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.077641 28108 raft_consensus.cc:729] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.077747 28108 consensus_queue.cc:260] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.078066 28108 ts_tablet_manager.cc:1398] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.078346 28112 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:38.078425 28112 ts_tablet_manager.cc:1367] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:38.078831 28112 raft_consensus.cc:348] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.078943 28112 raft_consensus.cc:374] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.078980 28112 raft_consensus.cc:729] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.079082 28112 consensus_queue.cc:260] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.079336 28112 ts_tablet_manager.cc:1398] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:38.081121 28107 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:38.081187 28107 ts_tablet_manager.cc:1367] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 01:58:38.081523 28107 raft_consensus.cc:348] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.081616 28107 raft_consensus.cc:374] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.081645 28107 raft_consensus.cc:729] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.081758 28107 consensus_queue.cc:260] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.082043 28107 ts_tablet_manager.cc:1398] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:38.082217 28107 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:38.082901 28107 tablet_bootstrap.cc:654] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.083859 28107 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:38.083935 28107 ts_tablet_manager.cc:1367] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:38.084381 28107 raft_consensus.cc:348] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.084478 28107 raft_consensus.cc:374] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.084523 28107 raft_consensus.cc:729] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.084641 28107 consensus_queue.cc:260] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.084858 28107 ts_tablet_manager.cc:1398] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:38.108811 28102 raft_consensus.cc:482] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:38.108889 28102 raft_consensus.cc:504] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.109208 28102 leader_election.cc:288] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:38.109490 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f84b6ffb074bc682aa7b087b8c3614" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:38.109513 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f84b6ffb074bc682aa7b087b8c3614" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:38.109586 26930 raft_consensus.cc:2454] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:38.109618 26660 raft_consensus.cc:2454] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:38.109879 26998 leader_election.cc:302] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:38.110049 28102 raft_consensus.cc:2781] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:38.110101 28102 raft_consensus.cc:482] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:38.110126 28102 raft_consensus.cc:3037] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.110777 28102 raft_consensus.cc:504] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.111039 28102 leader_election.cc:288] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:38.111310 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f84b6ffb074bc682aa7b087b8c3614" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:38.111399 26930 raft_consensus.cc:3037] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.111379 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f84b6ffb074bc682aa7b087b8c3614" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:38.111498 26660 raft_consensus.cc:3037] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.112068 26930 raft_consensus.cc:2454] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:38.112167 26660 raft_consensus.cc:2454] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:38.112282 26998 leader_election.cc:302] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:38.112434 28102 raft_consensus.cc:2781] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:38.112529 28102 raft_consensus.cc:686] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:38.112648 28102 consensus_queue.cc:237] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.113989 26510 catalog_manager.cc:5484] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:38.172168 28028 raft_consensus.cc:482] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:38.172256 28028 raft_consensus.cc:504] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.172614 28028 leader_election.cc:288] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:38.172920 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626774246b9445bc838427358e08f2cd" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:38.172930 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626774246b9445bc838427358e08f2cd" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:38.173038 26795 raft_consensus.cc:2454] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:38.173044 27064 raft_consensus.cc:2454] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:38.173262 26861 leader_election.cc:302] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:38.173437 28028 raft_consensus.cc:2781] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:38.173493 28028 raft_consensus.cc:482] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:38.173517 28028 raft_consensus.cc:3037] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.174288 28028 raft_consensus.cc:504] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.174602 28028 leader_election.cc:288] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:38.174892 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626774246b9445bc838427358e08f2cd" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:38.174893 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626774246b9445bc838427358e08f2cd" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:38.174993 26795 raft_consensus.cc:3037] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.174993 27064 raft_consensus.cc:3037] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.175648 26795 raft_consensus.cc:2454] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:38.175660 27064 raft_consensus.cc:2454] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:38.175897 26861 leader_election.cc:302] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:38.176043 28028 raft_consensus.cc:2781] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:38.176211 28028 raft_consensus.cc:686] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:38.176337 28028 consensus_queue.cc:237] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:38.177775 26510 catalog_manager.cc:5484] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:38.233834 27864 raft_consensus.cc:482] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:38.233937 27864 raft_consensus.cc:504] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.234297 27864 leader_election.cc:288] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:38.234561 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69df2cdb31d640ff9f43bb395f2f26fe" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:38.234635 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69df2cdb31d640ff9f43bb395f2f26fe" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:38.234676 26930 raft_consensus.cc:2454] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:38.234759 26795 raft_consensus.cc:2454] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:38.234910 26595 leader_election.cc:302] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:38.235074 27864 raft_consensus.cc:2781] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:38.235141 27864 raft_consensus.cc:482] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:38.235168 27864 raft_consensus.cc:3037] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.235958 27864 raft_consensus.cc:504] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.236246 27864 leader_election.cc:288] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:38.236483 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69df2cdb31d640ff9f43bb395f2f26fe" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:38.236537 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69df2cdb31d640ff9f43bb395f2f26fe" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:38.236577 26930 raft_consensus.cc:3037] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.236635 26795 raft_consensus.cc:3037] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.237231 26930 raft_consensus.cc:2454] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:38.237285 26795 raft_consensus.cc:2454] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:38.237478 26595 leader_election.cc:302] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:38.237640 27864 raft_consensus.cc:2781] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:38.237835 27864 raft_consensus.cc:686] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:38.237954 27864 consensus_queue.cc:237] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.239421 26510 catalog_manager.cc:5484] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
W20240502 01:58:38.353648 28126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:38.353849 28126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:38.356981 28126 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 01:58:38.424491 26660 raft_consensus.cc:1261] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.424811 26930 raft_consensus.cc:1261] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.425602 28081 consensus_queue.cc:1035] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.425890 28081 consensus_queue.cc:1035] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.429991 26929 raft_consensus.cc:1261] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.430902 26795 raft_consensus.cc:1261] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.430931 28123 consensus_queue.cc:1035] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.431160 27064 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.431849 28028 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.432691 28123 consensus_queue.cc:1035] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.433666 26795 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.438542 28121 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 93.4845 ms
time per row: 0.0467422 ms
I20240502 01:58:38.488133 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:38.492851 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 38c27ffed0ec4d2ca8525682b609369e (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.492970 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 38c27ffed0ec4d2ca8525682b609369e (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.493129 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c27ffed0ec4d2ca8525682b609369e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.493220 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c27ffed0ec4d2ca8525682b609369e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.493788 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 55fcea76c3c24decb635ffd912d0a738 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.493786 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 55fcea76c3c24decb635ffd912d0a738 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.493991 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55fcea76c3c24decb635ffd912d0a738. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.493999 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55fcea76c3c24decb635ffd912d0a738. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.495285 28108 tablet_bootstrap.cc:492] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:38.495421 26641 tablet_service.cc:1467] Processing CreateTablet for tablet cc7e7947fe8a466da4778d274184ac9c (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.495625 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc7e7947fe8a466da4778d274184ac9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.495851 28111 tablet_bootstrap.cc:492] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:38.496037 28108 tablet_bootstrap.cc:654] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.496618 28111 tablet_bootstrap.cc:654] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.497606 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 38c27ffed0ec4d2ca8525682b609369e (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.497694 28108 tablet_bootstrap.cc:492] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:38.497759 28108 ts_tablet_manager.cc:1367] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:38.497880 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c27ffed0ec4d2ca8525682b609369e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.498117 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 55fcea76c3c24decb635ffd912d0a738 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.498196 28108 raft_consensus.cc:348] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.498292 28108 raft_consensus.cc:374] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.498318 28108 raft_consensus.cc:729] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.498356 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55fcea76c3c24decb635ffd912d0a738. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.498418 28108 consensus_queue.cc:260] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.498497 27044 tablet_service.cc:1467] Processing CreateTablet for tablet cc7e7947fe8a466da4778d274184ac9c (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.498647 28108 ts_tablet_manager.cc:1398] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.498694 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc7e7947fe8a466da4778d274184ac9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.498791 28108 tablet_bootstrap.cc:492] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:38.499575 28108 tablet_bootstrap.cc:654] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.499974 28112 tablet_bootstrap.cc:492] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:38.500406 26907 tablet_service.cc:1467] Processing CreateTablet for tablet cc7e7947fe8a466da4778d274184ac9c (DEFAULT_TABLE table=rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:38.500604 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc7e7947fe8a466da4778d274184ac9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:38.500703 28112 tablet_bootstrap.cc:654] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.500802 28107 tablet_bootstrap.cc:492] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:38.501542 28107 tablet_bootstrap.cc:654] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.501731 28108 tablet_bootstrap.cc:492] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:38.501809 28108 ts_tablet_manager.cc:1367] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:38.502231 28108 raft_consensus.cc:348] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.502316 28108 raft_consensus.cc:374] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.502344 28108 raft_consensus.cc:729] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.502467 28108 consensus_queue.cc:260] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.502565 28107 tablet_bootstrap.cc:492] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:38.502635 28107 ts_tablet_manager.cc:1367] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:38.502810 28108 ts_tablet_manager.cc:1398] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.503008 28107 raft_consensus.cc:348] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.503121 28107 raft_consensus.cc:374] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.503155 28107 raft_consensus.cc:729] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.503263 28107 consensus_queue.cc:260] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.503551 28107 ts_tablet_manager.cc:1398] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.503747 28107 tablet_bootstrap.cc:492] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:38.504493 28112 tablet_bootstrap.cc:492] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:38.504547 28107 tablet_bootstrap.cc:654] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.504572 28112 ts_tablet_manager.cc:1367] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 01:58:38.504777 28111 tablet_bootstrap.cc:492] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:38.504844 28111 ts_tablet_manager.cc:1367] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 01:58:38.504977 28112 raft_consensus.cc:348] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.505062 28112 raft_consensus.cc:374] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.505089 28112 raft_consensus.cc:729] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.505189 28112 consensus_queue.cc:260] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.505218 28111 raft_consensus.cc:348] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.505312 28111 raft_consensus.cc:374] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.505340 28111 raft_consensus.cc:729] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.505445 28112 ts_tablet_manager.cc:1398] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:38.505447 28111 consensus_queue.cc:260] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.505595 28112 tablet_bootstrap.cc:492] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:38.505707 28111 ts_tablet_manager.cc:1398] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.505729 28107 tablet_bootstrap.cc:492] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:38.505798 28107 ts_tablet_manager.cc:1367] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:38.505865 28111 tablet_bootstrap.cc:492] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:38.506183 28107 raft_consensus.cc:348] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.506249 28107 raft_consensus.cc:374] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.506268 28107 raft_consensus.cc:729] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.506359 28107 consensus_queue.cc:260] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.506556 28112 tablet_bootstrap.cc:654] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.506598 28107 ts_tablet_manager.cc:1398] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.506650 28111 tablet_bootstrap.cc:654] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.507598 28111 tablet_bootstrap.cc:492] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:38.507675 28111 ts_tablet_manager.cc:1367] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:38.508076 28111 raft_consensus.cc:348] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.508178 28111 raft_consensus.cc:374] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.508212 28111 raft_consensus.cc:729] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.508625 28112 tablet_bootstrap.cc:492] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:38.508698 28112 ts_tablet_manager.cc:1367] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:38.509101 28112 raft_consensus.cc:348] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.509200 28112 raft_consensus.cc:374] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.509238 28112 raft_consensus.cc:729] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.509362 28112 consensus_queue.cc:260] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.509632 28112 ts_tablet_manager.cc:1398] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:38.509783 28111 consensus_queue.cc:260] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.510061 28111 ts_tablet_manager.cc:1398] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:38.510215 28111 tablet_bootstrap.cc:492] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:38.511016 28111 tablet_bootstrap.cc:654] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:38.512032 28111 tablet_bootstrap.cc:492] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:38.512115 28111 ts_tablet_manager.cc:1367] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:38.512431 28111 raft_consensus.cc:348] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.512508 28111 raft_consensus.cc:374] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:38.512542 28111 raft_consensus.cc:729] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:38.512647 28111 consensus_queue.cc:260] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.512878 28111 ts_tablet_manager.cc:1398] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:38.636523 28081 raft_consensus.cc:482] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:38.636643 28081 raft_consensus.cc:504] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.637005 28081 leader_election.cc:288] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:38.637269 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c27ffed0ec4d2ca8525682b609369e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:38.637338 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c27ffed0ec4d2ca8525682b609369e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:38.637392 26660 raft_consensus.cc:2454] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:38.637462 26794 raft_consensus.cc:2454] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:38.637663 26997 leader_election.cc:302] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:38.637822 28081 raft_consensus.cc:2781] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:38.637881 28081 raft_consensus.cc:482] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:38.637924 28081 raft_consensus.cc:3037] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.638703 28081 raft_consensus.cc:504] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.638984 28081 leader_election.cc:288] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:38.639268 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c27ffed0ec4d2ca8525682b609369e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:38.639341 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c27ffed0ec4d2ca8525682b609369e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:38.639389 26660 raft_consensus.cc:3037] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.639436 26794 raft_consensus.cc:3037] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.640256 26660 raft_consensus.cc:2454] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:38.640256 26794 raft_consensus.cc:2454] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:38.640517 26997 leader_election.cc:302] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:38.640658 28081 raft_consensus.cc:2781] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:38.640738 28081 raft_consensus.cc:686] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:38.640877 28081 consensus_queue.cc:237] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:38.642431 26506 catalog_manager.cc:5484] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:38.711539 28121 raft_consensus.cc:482] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:38.711627 28121 raft_consensus.cc:504] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.711944 28121 leader_election.cc:288] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:38.712194 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc7e7947fe8a466da4778d274184ac9c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:38.712234 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc7e7947fe8a466da4778d274184ac9c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:38.712317 27064 raft_consensus.cc:2454] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:38.712358 26660 raft_consensus.cc:2454] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:38.712569 26862 leader_election.cc:302] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:38.712713 28121 raft_consensus.cc:2781] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:38.712771 28121 raft_consensus.cc:482] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:38.712815 28121 raft_consensus.cc:3037] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.713554 28121 raft_consensus.cc:504] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.713842 28121 leader_election.cc:288] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:38.714143 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc7e7947fe8a466da4778d274184ac9c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:38.714157 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc7e7947fe8a466da4778d274184ac9c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:38.714257 26660 raft_consensus.cc:3037] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.714263 27064 raft_consensus.cc:3037] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.715116 27064 raft_consensus.cc:2454] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:38.715116 26660 raft_consensus.cc:2454] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:38.715374 26861 leader_election.cc:302] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:38.715534 28121 raft_consensus.cc:2781] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:38.715612 28121 raft_consensus.cc:686] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:38.715729 28121 consensus_queue.cc:237] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:38.717214 26506 catalog_manager.cc:5484] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:38.745013 28123 raft_consensus.cc:482] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:38.745098 28123 raft_consensus.cc:504] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.745412 28123 leader_election.cc:288] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:38.745676 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55fcea76c3c24decb635ffd912d0a738" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:38.745697 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55fcea76c3c24decb635ffd912d0a738" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:38.745782 26930 raft_consensus.cc:2454] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:38.745795 26794 raft_consensus.cc:2454] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:38.746062 26595 leader_election.cc:302] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:38.746214 28123 raft_consensus.cc:2781] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:38.746271 28123 raft_consensus.cc:482] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:38.746312 28123 raft_consensus.cc:3037] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.746943 28123 raft_consensus.cc:504] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.747225 28123 leader_election.cc:288] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:38.747478 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55fcea76c3c24decb635ffd912d0a738" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:38.747524 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55fcea76c3c24decb635ffd912d0a738" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:38.747573 26794 raft_consensus.cc:3037] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.747603 26930 raft_consensus.cc:3037] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:38.748227 26930 raft_consensus.cc:2454] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:38.748258 26794 raft_consensus.cc:2454] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:38.748469 26595 leader_election.cc:302] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:38.748617 28123 raft_consensus.cc:2781] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:38.748704 28123 raft_consensus.cc:686] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:38.748826 28123 consensus_queue.cc:237] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:38.750340 26506 catalog_manager.cc:5484] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:38.883666 28182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:38.883847 28182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:38.886857 28182 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 01:58:38.978675 27064 raft_consensus.cc:1261] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.979106 26660 raft_consensus.cc:1261] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.979743 28121 consensus_queue.cc:1035] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.980069 28028 consensus_queue.cc:1035] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.985280 26794 raft_consensus.cc:1261] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.985554 26659 raft_consensus.cc:1261] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.986124 28102 consensus_queue.cc:1035] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.986413 28102 consensus_queue.cc:1035] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.991391 26795 raft_consensus.cc:1261] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.991391 26930 raft_consensus.cc:1261] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:38.991865 28168 consensus_queue.cc:1035] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:38.992574 28168 consensus_queue.cc:1035] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 121.143 ms
time per row: 0.0605713 ms
I20240502 01:58:39.045718 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:39.050911 26907 tablet_service.cc:1467] Processing CreateTablet for tablet ae9f78db63b74beca267ff5f12a211ae (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.051234 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae9f78db63b74beca267ff5f12a211ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.051247 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 3d53d3130fa84270ad934353a5dbab41 (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.051451 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d53d3130fa84270ad934353a5dbab41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.051515 26774 tablet_service.cc:1467] Processing CreateTablet for tablet ae9f78db63b74beca267ff5f12a211ae (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.051735 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae9f78db63b74beca267ff5f12a211ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.053175 28247 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:39.053555 27044 tablet_service.cc:1467] Processing CreateTablet for tablet ae9f78db63b74beca267ff5f12a211ae (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.053803 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae9f78db63b74beca267ff5f12a211ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.054100 28247 tablet_bootstrap.cc:654] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.054358 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 49ac3a00934944bdb5c9e440045532a3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.054551 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ac3a00934944bdb5c9e440045532a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.055104 28247 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:39.055176 28247 ts_tablet_manager.cc:1367] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.055572 28247 raft_consensus.cc:348] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.055626 28250 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:39.055656 28247 raft_consensus.cc:374] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.055685 28247 raft_consensus.cc:729] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.055828 28247 consensus_queue.cc:260] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.055931 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 49ac3a00934944bdb5c9e440045532a3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.056097 28247 ts_tablet_manager.cc:1398] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.056182 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ac3a00934944bdb5c9e440045532a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.056399 28250 tablet_bootstrap.cc:654] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.056723 28247 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:39.057324 28250 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:39.057387 28247 tablet_bootstrap.cc:654] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.057394 28250 ts_tablet_manager.cc:1367] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.057791 28250 raft_consensus.cc:348] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.057883 28250 raft_consensus.cc:374] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.057910 28250 raft_consensus.cc:729] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.058063 28250 consensus_queue.cc:260] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.058204 28247 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:39.058269 28247 ts_tablet_manager.cc:1367] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.058346 28250 ts_tablet_manager.cc:1398] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.058576 28253 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:39.058578 28247 raft_consensus.cc:348] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.058670 28247 raft_consensus.cc:374] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.058699 28247 raft_consensus.cc:729] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.058993 28247 consensus_queue.cc:260] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.059418 28247 ts_tablet_manager.cc:1398] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.059561 28247 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:39.060166 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 49ac3a00934944bdb5c9e440045532a3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.059950 28253 tablet_bootstrap.cc:654] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.060400 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ac3a00934944bdb5c9e440045532a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.061014 28247 tablet_bootstrap.cc:654] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.061575 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 3d53d3130fa84270ad934353a5dbab41 (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.061739 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d53d3130fa84270ad934353a5dbab41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.062583 28250 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:39.062813 28247 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:39.062892 28247 ts_tablet_manager.cc:1367] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:39.063251 28247 raft_consensus.cc:348] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.063374 28250 tablet_bootstrap.cc:654] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.063400 28247 raft_consensus.cc:374] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.063436 28247 raft_consensus.cc:729] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.063587 28247 consensus_queue.cc:260] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.063920 28247 ts_tablet_manager.cc:1398] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:39.064446 28250 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:39.064567 28250 ts_tablet_manager.cc:1367] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:39.064585 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 3d53d3130fa84270ad934353a5dbab41 (DEFAULT_TABLE table=rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.064824 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d53d3130fa84270ad934353a5dbab41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.064970 28250 raft_consensus.cc:348] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.065048 28250 raft_consensus.cc:374] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.065073 28250 raft_consensus.cc:729] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.065205 28250 consensus_queue.cc:260] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.065487 28250 ts_tablet_manager.cc:1398] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.065972 28250 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:39.066695 28248 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:39.066936 28250 tablet_bootstrap.cc:654] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.067523 28248 tablet_bootstrap.cc:654] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.068261 28250 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:39.068344 28250 ts_tablet_manager.cc:1367] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:39.068485 28248 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:39.068567 28248 ts_tablet_manager.cc:1367] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.068724 28250 raft_consensus.cc:348] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.068814 28250 raft_consensus.cc:374] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.068835 28250 raft_consensus.cc:729] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.068925 28250 consensus_queue.cc:260] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.068928 28248 raft_consensus.cc:348] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.069017 28248 raft_consensus.cc:374] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.069068 28248 raft_consensus.cc:729] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.069164 28250 ts_tablet_manager.cc:1398] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:39.069190 28248 consensus_queue.cc:260] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.069432 28248 ts_tablet_manager.cc:1398] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.069592 28248 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:39.071324 28248 tablet_bootstrap.cc:654] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.072177 28253 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:39.072263 28253 ts_tablet_manager.cc:1367] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:39.072703 28253 raft_consensus.cc:348] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.072814 28253 raft_consensus.cc:374] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.072858 28253 raft_consensus.cc:729] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.072976 28253 consensus_queue.cc:260] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.073231 28253 ts_tablet_manager.cc:1398] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:39.073810 28248 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:39.073892 28248 ts_tablet_manager.cc:1367] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:39.074457 28248 raft_consensus.cc:348] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.074553 28248 raft_consensus.cc:374] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.074582 28248 raft_consensus.cc:729] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.074728 28248 consensus_queue.cc:260] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.074966 28248 ts_tablet_manager.cc:1398] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.088534 28081 raft_consensus.cc:482] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:39.088610 28081 raft_consensus.cc:504] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.088893 28081 leader_election.cc:288] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.089161 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d53d3130fa84270ad934353a5dbab41" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:39.089203 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d53d3130fa84270ad934353a5dbab41" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:39.089279 26794 raft_consensus.cc:2454] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:39.089318 26930 raft_consensus.cc:2454] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:39.089561 26998 leader_election.cc:302] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:39.089709 28081 raft_consensus.cc:2781] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:39.089764 28081 raft_consensus.cc:482] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:39.089818 28081 raft_consensus.cc:3037] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.090546 28081 raft_consensus.cc:504] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.090843 28081 leader_election.cc:288] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.091086 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d53d3130fa84270ad934353a5dbab41" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:39.091132 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d53d3130fa84270ad934353a5dbab41" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:39.091173 26794 raft_consensus.cc:3037] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.091208 26930 raft_consensus.cc:3037] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.091809 26794 raft_consensus.cc:2454] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:39.091833 26930 raft_consensus.cc:2454] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:39.092094 26998 leader_election.cc:302] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:39.092247 28081 raft_consensus.cc:2781] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:39.092324 28081 raft_consensus.cc:686] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:39.092442 28081 consensus_queue.cc:237] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.093824 26509 catalog_manager.cc:5484] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:39.229132 28081 raft_consensus.cc:482] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:39.229249 28081 raft_consensus.cc:504] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.229602 28081 leader_election.cc:288] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.229908 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae9f78db63b74beca267ff5f12a211ae" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:39.229943 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae9f78db63b74beca267ff5f12a211ae" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:39.230044 26930 raft_consensus.cc:2454] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:39.230090 26794 raft_consensus.cc:2454] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:39.230286 26998 leader_election.cc:302] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:39.230463 28081 raft_consensus.cc:2781] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:39.230515 28081 raft_consensus.cc:482] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:39.230543 28081 raft_consensus.cc:3037] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.231302 28081 raft_consensus.cc:504] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.231595 28081 leader_election.cc:288] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.231868 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae9f78db63b74beca267ff5f12a211ae" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:39.231881 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae9f78db63b74beca267ff5f12a211ae" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:39.231972 26930 raft_consensus.cc:3037] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.231972 26794 raft_consensus.cc:3037] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.232822 26930 raft_consensus.cc:2454] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:39.232818 26794 raft_consensus.cc:2454] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:39.233076 26997 leader_election.cc:302] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:39.233238 28081 raft_consensus.cc:2781] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:39.233410 28081 raft_consensus.cc:686] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:39.233584 28081 consensus_queue.cc:237] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.235548 26509 catalog_manager.cc:5484] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:39.247941 28236 raft_consensus.cc:482] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:39.248014 28236 raft_consensus.cc:504] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.248301 28236 leader_election.cc:288] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:39.248607 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ac3a00934944bdb5c9e440045532a3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:39.248610 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ac3a00934944bdb5c9e440045532a3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:39.248726 27064 raft_consensus.cc:2454] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:39.248726 26659 raft_consensus.cc:2454] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:39.248955 26729 leader_election.cc:302] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:39.249150 28236 raft_consensus.cc:2781] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:39.249203 28236 raft_consensus.cc:482] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:39.249228 28236 raft_consensus.cc:3037] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.249891 28236 raft_consensus.cc:504] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.250221 28236 leader_election.cc:288] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:39.250494 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ac3a00934944bdb5c9e440045532a3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:39.250578 27064 raft_consensus.cc:3037] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.251195 27064 raft_consensus.cc:2454] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:39.251402 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49ac3a00934944bdb5c9e440045532a3" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:39.251425 26730 leader_election.cc:302] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:39.251513 26659 raft_consensus.cc:3037] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.251628 28236 raft_consensus.cc:2781] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:39.251710 28236 raft_consensus.cc:686] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:39.251832 28236 consensus_queue.cc:237] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.252274 26659 raft_consensus.cc:2454] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:39.253386 26509 catalog_manager.cc:5484] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
W20240502 01:58:39.394469 28263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:39.394661 28263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:39.397825 28263 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 01:58:39.463933 26660 raft_consensus.cc:1261] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.464354 28236 consensus_queue.cc:1035] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.468451 26930 raft_consensus.cc:1261] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.468500 26929 raft_consensus.cc:1261] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.468868 28225 mvcc.cc:205] Tried to move back new op lower bound from 7023063529319731200 to 7023063528456589312. Current Snapshot: MvccSnapshot[applied={T|T < 7023063529319731200}]
I20240502 01:58:39.470001 26795 raft_consensus.cc:1261] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.470554 26794 raft_consensus.cc:1261] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.471746 27064 raft_consensus.cc:1261] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.472330 28081 consensus_queue.cc:1035] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.472395 28102 consensus_queue.cc:1035] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.473006 28291 consensus_queue.cc:1035] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.473277 28291 consensus_queue.cc:1035] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.473348 28105 consensus_queue.cc:1035] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.484862 28218 mvcc.cc:205] Tried to move back new op lower bound from 7023063529331511296 to 7023063528382193664. Current Snapshot: MvccSnapshot[applied={T|T < 7023063529331511296}]
INSERT report
rows total: 2000
time total: 95.5305 ms
time per row: 0.0477653 ms
I20240502 01:58:39.529367 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:39.534122 26774 tablet_service.cc:1467] Processing CreateTablet for tablet b67e582a01b442f591d36621aa6cdc68 (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.534389 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67e582a01b442f591d36621aa6cdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.534389 26907 tablet_service.cc:1467] Processing CreateTablet for tablet b67e582a01b442f591d36621aa6cdc68 (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.534637 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67e582a01b442f591d36621aa6cdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.534896 26641 tablet_service.cc:1467] Processing CreateTablet for tablet b67e582a01b442f591d36621aa6cdc68 (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.535123 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67e582a01b442f591d36621aa6cdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.536090 28247 tablet_bootstrap.cc:492] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:39.536528 28248 tablet_bootstrap.cc:492] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:39.536698 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 8c76b55d6462415d89dd0c591759a3be (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.536824 28247 tablet_bootstrap.cc:654] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.536906 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c76b55d6462415d89dd0c591759a3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.537204 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6a87dff021447da6224891638c039a (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.537325 28248 tablet_bootstrap.cc:654] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.537386 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e6a87dff021447da6224891638c039a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.538200 28253 tablet_bootstrap.cc:492] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:39.538278 28247 tablet_bootstrap.cc:492] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:39.538278 28248 tablet_bootstrap.cc:492] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:39.538358 28248 ts_tablet_manager.cc:1367] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.538368 28247 ts_tablet_manager.cc:1367] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:39.538760 28248 raft_consensus.cc:348] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.538760 28247 raft_consensus.cc:348] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.538857 28248 raft_consensus.cc:374] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.538857 28247 raft_consensus.cc:374] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.538818 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6a87dff021447da6224891638c039a (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.538889 28248 raft_consensus.cc:729] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.538889 28247 raft_consensus.cc:729] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.538980 28253 tablet_bootstrap.cc:654] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.538990 28247 consensus_queue.cc:260] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.538990 28248 consensus_queue.cc:260] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.539072 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e6a87dff021447da6224891638c039a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.539244 28247 ts_tablet_manager.cc:1398] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:39.539245 28248 ts_tablet_manager.cc:1398] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:39.539393 28248 tablet_bootstrap.cc:492] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:39.540035 28248 tablet_bootstrap.cc:654] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.540413 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 8c76b55d6462415d89dd0c591759a3be (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.540505 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 8c76b55d6462415d89dd0c591759a3be (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.540637 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c76b55d6462415d89dd0c591759a3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.540707 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c76b55d6462415d89dd0c591759a3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.542536 28250 tablet_bootstrap.cc:492] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:39.543262 28250 tablet_bootstrap.cc:654] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.543956 28248 tablet_bootstrap.cc:492] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:39.544035 28248 ts_tablet_manager.cc:1367] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 01:58:39.544015 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6a87dff021447da6224891638c039a (DEFAULT_TABLE table=rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:39.544216 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e6a87dff021447da6224891638c039a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:39.544225 28247 tablet_bootstrap.cc:492] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:39.544447 28248 raft_consensus.cc:348] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.544531 28248 raft_consensus.cc:374] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.544557 28248 raft_consensus.cc:729] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.544663 28248 consensus_queue.cc:260] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.544874 28247 tablet_bootstrap.cc:654] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.544909 28248 ts_tablet_manager.cc:1398] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:39.545058 28248 tablet_bootstrap.cc:492] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:39.545761 28250 tablet_bootstrap.cc:492] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:39.545856 28250 ts_tablet_manager.cc:1367] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:39.546303 28250 raft_consensus.cc:348] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.546406 28250 raft_consensus.cc:374] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.546440 28250 raft_consensus.cc:729] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.546533 28250 consensus_queue.cc:260] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.546727 28248 tablet_bootstrap.cc:654] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.546737 28250 ts_tablet_manager.cc:1398] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.547981 28248 tablet_bootstrap.cc:492] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:39.548065 28248 ts_tablet_manager.cc:1367] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:39.548460 28248 raft_consensus.cc:348] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.548552 28248 raft_consensus.cc:374] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.548583 28248 raft_consensus.cc:729] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.548686 28248 consensus_queue.cc:260] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.548894 28253 tablet_bootstrap.cc:492] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:39.548925 28248 ts_tablet_manager.cc:1398] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.548976 28253 ts_tablet_manager.cc:1367] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:58:39.549362 28253 raft_consensus.cc:348] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.549449 28253 raft_consensus.cc:374] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.549477 28253 raft_consensus.cc:729] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.549582 28253 consensus_queue.cc:260] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.549620 28247 tablet_bootstrap.cc:492] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:39.549683 28247 ts_tablet_manager.cc:1367] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 01:58:39.549830 28253 ts_tablet_manager.cc:1398] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.549979 28253 tablet_bootstrap.cc:492] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:39.550192 28247 raft_consensus.cc:348] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.550290 28247 raft_consensus.cc:374] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.550326 28247 raft_consensus.cc:729] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.550434 28247 consensus_queue.cc:260] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.550659 28247 ts_tablet_manager.cc:1398] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.550683 28253 tablet_bootstrap.cc:654] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.551627 28253 tablet_bootstrap.cc:492] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:39.551699 28253 ts_tablet_manager.cc:1367] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.552006 28253 raft_consensus.cc:348] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.552101 28253 raft_consensus.cc:374] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.552132 28253 raft_consensus.cc:729] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.552233 28253 consensus_queue.cc:260] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.552541 28253 ts_tablet_manager.cc:1398] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.552695 28253 tablet_bootstrap.cc:492] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:39.553268 28253 tablet_bootstrap.cc:654] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:39.554198 28253 tablet_bootstrap.cc:492] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:39.554270 28253 ts_tablet_manager.cc:1367] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:39.554607 28253 raft_consensus.cc:348] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.554710 28253 raft_consensus.cc:374] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:39.554742 28253 raft_consensus.cc:729] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:39.554857 28253 consensus_queue.cc:260] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.555102 28253 ts_tablet_manager.cc:1398] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:39.608304 28081 raft_consensus.cc:482] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:39.608397 28081 raft_consensus.cc:504] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.608692 28081 leader_election.cc:288] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.609004 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6a87dff021447da6224891638c039a" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:39.609040 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6a87dff021447da6224891638c039a" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:39.609122 26660 raft_consensus.cc:2454] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:39.609145 26929 raft_consensus.cc:2454] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:39.609400 26997 leader_election.cc:302] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:39.609553 28081 raft_consensus.cc:2781] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:39.609617 28081 raft_consensus.cc:482] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:39.609658 28081 raft_consensus.cc:3037] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.610394 28081 raft_consensus.cc:504] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.610667 28081 leader_election.cc:288] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.610973 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6a87dff021447da6224891638c039a" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:39.610985 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6a87dff021447da6224891638c039a" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:39.611064 26929 raft_consensus.cc:3037] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.611076 26660 raft_consensus.cc:3037] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.611702 26660 raft_consensus.cc:2454] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:39.611711 26929 raft_consensus.cc:2454] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:39.611974 26998 leader_election.cc:302] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:39.612112 28081 raft_consensus.cc:2781] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:39.612198 28081 raft_consensus.cc:686] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:39.612331 28081 consensus_queue.cc:237] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:39.613838 26510 catalog_manager.cc:5484] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:39.663977 28242 raft_consensus.cc:482] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:39.664062 28242 raft_consensus.cc:504] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.664426 28242 leader_election.cc:288] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:39.664670 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c76b55d6462415d89dd0c591759a3be" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:39.664786 26929 raft_consensus.cc:2454] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:39.664770 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c76b55d6462415d89dd0c591759a3be" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:39.664892 26795 raft_consensus.cc:2454] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:39.665022 26595 leader_election.cc:302] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:39.665185 28242 raft_consensus.cc:2781] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:39.665242 28242 raft_consensus.cc:482] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:39.665282 28242 raft_consensus.cc:3037] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.666046 28242 raft_consensus.cc:504] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.666383 28242 leader_election.cc:288] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:39.666652 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c76b55d6462415d89dd0c591759a3be" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:39.666755 26929 raft_consensus.cc:3037] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.666728 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c76b55d6462415d89dd0c591759a3be" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:39.666836 26795 raft_consensus.cc:3037] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.667611 26929 raft_consensus.cc:2454] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:39.667693 26795 raft_consensus.cc:2454] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:39.667867 26595 leader_election.cc:302] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:39.668081 28242 raft_consensus.cc:2781] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:39.668223 28242 raft_consensus.cc:686] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:39.668347 28242 consensus_queue.cc:237] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:39.669833 26510 catalog_manager.cc:5484] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:39.753000 28297 raft_consensus.cc:482] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:39.753084 28297 raft_consensus.cc:504] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.753407 28297 leader_election.cc:288] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.753641 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67e582a01b442f591d36621aa6cdc68" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:39.753700 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67e582a01b442f591d36621aa6cdc68" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:39.753764 26660 raft_consensus.cc:2454] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:39.753826 26929 raft_consensus.cc:2454] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:39.754011 26729 leader_election.cc:302] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:39.754187 28105 raft_consensus.cc:2781] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:39.754247 28105 raft_consensus.cc:482] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:39.754292 28105 raft_consensus.cc:3037] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.755064 28105 raft_consensus.cc:504] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.755349 28105 leader_election.cc:288] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:39.755824 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67e582a01b442f591d36621aa6cdc68" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:39.755823 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67e582a01b442f591d36621aa6cdc68" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:39.755918 26929 raft_consensus.cc:3037] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.755923 26659 raft_consensus.cc:3037] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:39.756623 26929 raft_consensus.cc:2454] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:39.756628 26659 raft_consensus.cc:2454] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:39.756850 26730 leader_election.cc:302] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:39.757007 28105 raft_consensus.cc:2781] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:39.757086 28105 raft_consensus.cc:686] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:39.757225 28105 consensus_queue.cc:237] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:39.758706 26510 catalog_manager.cc:5484] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:39.875855 28315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:39.876051 28315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:39.879226 28315 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_10'
I20240502 01:58:39.964447 26929 raft_consensus.cc:1261] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.964887 28297 consensus_queue.cc:1035] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.965509 26659 raft_consensus.cc:1261] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.965977 28297 consensus_queue.cc:1035] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.972788 26930 raft_consensus.cc:1261] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.973258 28102 consensus_queue.cc:1035] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.973376 26660 raft_consensus.cc:1261] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.973732 28102 consensus_queue.cc:1035] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.988092 28227 mvcc.cc:205] Tried to move back new op lower bound from 7023063531416313856 to 7023063530526756864. Current Snapshot: MvccSnapshot[applied={T|T < 7023063531416313856}]
I20240502 01:58:39.992967 26795 raft_consensus.cc:1261] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.993340 28242 consensus_queue.cc:1035] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:39.999202 26929 raft_consensus.cc:1261] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:39.999716 28242 consensus_queue.cc:1035] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:40.013139 28299 mvcc.cc:205] Tried to move back new op lower bound from 7023063531401973760 to 7023063529933299712. Current Snapshot: MvccSnapshot[applied={T|T < 7023063531401973760 or (T in {7023063531401973760})}]
INSERT report
rows total: 2000
time total: 126.92 ms
time per row: 0.0634598 ms
I20240502 01:58:40.044219 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:40.049430 27044 tablet_service.cc:1467] Processing CreateTablet for tablet cdd32c8294b44540ac5b832cc48d2519 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.049553 26641 tablet_service.cc:1467] Processing CreateTablet for tablet cdd32c8294b44540ac5b832cc48d2519 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.049721 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdd32c8294b44540ac5b832cc48d2519. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.049773 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdd32c8294b44540ac5b832cc48d2519. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.050184 26774 tablet_service.cc:1467] Processing CreateTablet for tablet cdd32c8294b44540ac5b832cc48d2519 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.050426 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdd32c8294b44540ac5b832cc48d2519. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.051798 28253 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:40.052270 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 5086929a32c34548b9376730565c0061 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.052484 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5086929a32c34548b9376730565c0061. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.052589 28253 tablet_bootstrap.cc:654] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.052635 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 5086929a32c34548b9376730565c0061 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.052837 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5086929a32c34548b9376730565c0061. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.053400 26909 tablet_service.cc:1467] Processing CreateTablet for tablet e1cd4681de8645ff92b479ccdd3eac37 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.053589 28355 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:40.053665 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1cd4681de8645ff92b479ccdd3eac37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.054422 28355 tablet_bootstrap.cc:654] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.054852 27044 tablet_service.cc:1467] Processing CreateTablet for tablet e1cd4681de8645ff92b479ccdd3eac37 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.055073 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1cd4681de8645ff92b479ccdd3eac37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.055217 28354 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:40.055671 28358 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:40.053400 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 5086929a32c34548b9376730565c0061 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.055908 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5086929a32c34548b9376730565c0061. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.055996 28354 tablet_bootstrap.cc:654] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.056921 28253 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:40.057001 28253 ts_tablet_manager.cc:1367] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:40.057438 28253 raft_consensus.cc:348] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.057493 26641 tablet_service.cc:1467] Processing CreateTablet for tablet e1cd4681de8645ff92b479ccdd3eac37 (DEFAULT_TABLE table=rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.057547 28253 raft_consensus.cc:374] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.057582 28253 raft_consensus.cc:729] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.057688 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1cd4681de8645ff92b479ccdd3eac37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.057687 28253 consensus_queue.cc:260] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.058077 28253 ts_tablet_manager.cc:1398] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.058214 28355 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:40.058321 28355 ts_tablet_manager.cc:1367] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:40.058646 28355 raft_consensus.cc:348] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.058727 28355 raft_consensus.cc:374] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.058755 28355 raft_consensus.cc:729] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.058879 28355 consensus_queue.cc:260] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.059206 28355 ts_tablet_manager.cc:1398] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:40.059440 28355 tablet_bootstrap.cc:492] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:40.059780 28253 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:40.060240 28355 tablet_bootstrap.cc:654] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.058153 28354 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:40.060580 28253 tablet_bootstrap.cc:654] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.060644 28354 ts_tablet_manager.cc:1367] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:40.061015 28354 raft_consensus.cc:348] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.061120 28354 raft_consensus.cc:374] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.061156 28354 raft_consensus.cc:729] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.061254 28354 consensus_queue.cc:260] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.061630 28358 tablet_bootstrap.cc:654] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.062108 28354 ts_tablet_manager.cc:1398] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.062273 28354 tablet_bootstrap.cc:492] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:40.062638 28355 tablet_bootstrap.cc:492] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:40.062709 28355 ts_tablet_manager.cc:1367] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:40.062827 28253 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:40.062901 28253 ts_tablet_manager.cc:1367] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:40.063009 28354 tablet_bootstrap.cc:654] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.063112 28355 raft_consensus.cc:348] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.063199 28355 raft_consensus.cc:374] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.063227 28355 raft_consensus.cc:729] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.063285 28253 raft_consensus.cc:348] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.063331 28355 consensus_queue.cc:260] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.063373 28253 raft_consensus.cc:374] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.063406 28253 raft_consensus.cc:729] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.063531 28253 consensus_queue.cc:260] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.063578 28355 ts_tablet_manager.cc:1398] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:58:40.063788 28253 ts_tablet_manager.cc:1398] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.064085 28358 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:40.064164 28358 ts_tablet_manager.cc:1367] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:40.064561 28358 raft_consensus.cc:348] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.064672 28358 raft_consensus.cc:374] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.064733 28358 raft_consensus.cc:729] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.064878 28358 consensus_queue.cc:260] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.065145 28354 tablet_bootstrap.cc:492] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:40.065153 28358 ts_tablet_manager.cc:1398] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.065253 28354 ts_tablet_manager.cc:1367] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:40.065301 28358 tablet_bootstrap.cc:492] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:40.065662 28354 raft_consensus.cc:348] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.065768 28354 raft_consensus.cc:374] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.065797 28354 raft_consensus.cc:729] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.066001 28354 consensus_queue.cc:260] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.066143 28358 tablet_bootstrap.cc:654] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.066291 28354 ts_tablet_manager.cc:1398] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.066435 28354 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:40.068532 28354 tablet_bootstrap.cc:654] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.069428 28358 tablet_bootstrap.cc:492] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:40.069505 28354 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:40.069573 28354 ts_tablet_manager.cc:1367] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:40.069512 28358 ts_tablet_manager.cc:1367] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:40.069942 28354 raft_consensus.cc:348] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.069980 28358 raft_consensus.cc:348] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.070056 28354 raft_consensus.cc:374] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.070081 28358 raft_consensus.cc:374] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.070087 28354 raft_consensus.cc:729] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.070118 28358 raft_consensus.cc:729] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.070191 28354 consensus_queue.cc:260] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.070441 28354 ts_tablet_manager.cc:1398] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:40.070468 28358 consensus_queue.cc:260] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.070724 28242 raft_consensus.cc:482] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:40.070796 28242 raft_consensus.cc:504] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.070909 28358 ts_tablet_manager.cc:1398] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.071203 28242 leader_election.cc:288] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:40.071962 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1cd4681de8645ff92b479ccdd3eac37" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:40.072072 26930 raft_consensus.cc:2454] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:40.072402 26595 leader_election.cc:302] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:40.072599 28242 raft_consensus.cc:2781] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:40.072656 28242 raft_consensus.cc:482] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:40.072698 28242 raft_consensus.cc:3037] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.073491 28242 raft_consensus.cc:504] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.073820 28242 leader_election.cc:288] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:40.074337 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1cd4681de8645ff92b479ccdd3eac37" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:40.074424 26930 raft_consensus.cc:3037] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.075271 26930 raft_consensus.cc:2454] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:40.075642 26595 leader_election.cc:302] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:40.075924 28242 raft_consensus.cc:2781] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:40.076010 28242 raft_consensus.cc:686] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:40.076030 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1cd4681de8645ff92b479ccdd3eac37" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:40.076078 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1cd4681de8645ff92b479ccdd3eac37" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:40.076133 27064 raft_consensus.cc:2454] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:40.076135 28242 consensus_queue.cc:237] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.077823 26510 catalog_manager.cc:5484] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:40.194491 28297 raft_consensus.cc:482] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:40.194588 28297 raft_consensus.cc:504] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.194941 28297 leader_election.cc:288] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:40.195212 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdd32c8294b44540ac5b832cc48d2519" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:40.195236 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdd32c8294b44540ac5b832cc48d2519" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:40.195351 26659 raft_consensus.cc:2454] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:40.195351 27063 raft_consensus.cc:2454] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:40.195585 26729 leader_election.cc:302] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:40.195740 28297 raft_consensus.cc:2781] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:40.195808 28297 raft_consensus.cc:482] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:40.195834 28297 raft_consensus.cc:3037] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.196604 28297 raft_consensus.cc:504] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.196909 28297 leader_election.cc:288] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:40.197181 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdd32c8294b44540ac5b832cc48d2519" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:40.197202 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdd32c8294b44540ac5b832cc48d2519" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:40.197290 27063 raft_consensus.cc:3037] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.197306 26659 raft_consensus.cc:3037] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.198006 27063 raft_consensus.cc:2454] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:40.198051 26659 raft_consensus.cc:2454] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:40.198284 26730 leader_election.cc:302] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:40.198436 28297 raft_consensus.cc:2781] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:40.198527 28297 raft_consensus.cc:686] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:40.198655 28297 consensus_queue.cc:237] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.201233 26513 catalog_manager.cc:5484] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:40.315493 28028 raft_consensus.cc:482] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:40.315591 28028 raft_consensus.cc:504] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.315937 28028 leader_election.cc:288] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:40.316259 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5086929a32c34548b9376730565c0061" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:40.316282 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5086929a32c34548b9376730565c0061" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:40.316413 26793 raft_consensus.cc:2454] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:40.316426 27063 raft_consensus.cc:2454] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:40.316668 26862 leader_election.cc:302] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:40.316841 28028 raft_consensus.cc:2781] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:40.316892 28028 raft_consensus.cc:482] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:40.316927 28028 raft_consensus.cc:3037] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.317659 28028 raft_consensus.cc:504] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.317953 28028 leader_election.cc:288] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:40.318257 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5086929a32c34548b9376730565c0061" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:40.318372 26793 raft_consensus.cc:3037] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.318349 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5086929a32c34548b9376730565c0061" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:40.318441 27063 raft_consensus.cc:3037] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.319293 27063 raft_consensus.cc:2454] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:40.319299 26793 raft_consensus.cc:2454] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:40.319576 26862 leader_election.cc:302] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:40.319746 28028 raft_consensus.cc:2781] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:40.319900 28028 raft_consensus.cc:686] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:40.320030 28028 consensus_queue.cc:237] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.322525 26513 catalog_manager.cc:5484] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
W20240502 01:58:40.451326 28371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:40.451542 28371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:40.454526 28371 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_11'
I20240502 01:58:40.534807 27063 raft_consensus.cc:1261] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:40.535209 28105 consensus_queue.cc:1035] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:40.537812 26930 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:40.538110 27064 raft_consensus.cc:3037] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.539152 27064 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:40.539981 26659 raft_consensus.cc:1261] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:40.540309 28242 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:40.540541 28105 consensus_queue.cc:1035] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:40.540827 28312 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:40.582648 27063 raft_consensus.cc:1261] T 5086929a32c34548b9376730565c0061 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:40.582810 26793 raft_consensus.cc:1261] T 5086929a32c34548b9376730565c0061 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:40.583034 28369 consensus_queue.cc:1035] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:40.583295 28420 consensus_queue.cc:1035] T 5086929a32c34548b9376730565c0061 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 119.004 ms
time per row: 0.0595018 ms
I20240502 01:58:40.612493 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:40.617455 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 2adcab4321d549919938465f639c9ddc (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.617540 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 2adcab4321d549919938465f639c9ddc (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.617730 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2adcab4321d549919938465f639c9ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.617815 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2adcab4321d549919938465f639c9ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.619455 28428 tablet_bootstrap.cc:492] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:40.619508 28429 tablet_bootstrap.cc:492] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:40.619849 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 2adcab4321d549919938465f639c9ddc (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.620106 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2adcab4321d549919938465f639c9ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.620371 28428 tablet_bootstrap.cc:654] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.620414 28429 tablet_bootstrap.cc:654] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.620450 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.620651 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1417ea39bb2b4716bfb3d11c05853c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.621850 28432 tablet_bootstrap.cc:492] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:40.622269 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1f07f16b0f49b4b3c3cc5511604532 (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.622483 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1f07f16b0f49b4b3c3cc5511604532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.623020 28432 tablet_bootstrap.cc:654] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.623329 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1f07f16b0f49b4b3c3cc5511604532 (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.623373 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.623584 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1f07f16b0f49b4b3c3cc5511604532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.624028 28432 tablet_bootstrap.cc:492] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:40.624104 28432 ts_tablet_manager.cc:1367] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:40.624125 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1f07f16b0f49b4b3c3cc5511604532 (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.624325 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1f07f16b0f49b4b3c3cc5511604532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.624516 28432 raft_consensus.cc:348] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.624608 28432 raft_consensus.cc:374] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.624636 28432 raft_consensus.cc:729] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.624744 28432 consensus_queue.cc:260] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.624830 28428 tablet_bootstrap.cc:492] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:40.624899 28428 ts_tablet_manager.cc:1367] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:40.624991 28432 ts_tablet_manager.cc:1398] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.625290 28428 raft_consensus.cc:348] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.625385 28428 raft_consensus.cc:374] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.625414 28428 raft_consensus.cc:729] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.625442 28429 tablet_bootstrap.cc:492] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:40.625516 28429 ts_tablet_manager.cc:1367] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:40.625527 28428 consensus_queue.cc:260] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.625799 28428 ts_tablet_manager.cc:1398] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:40.625921 28429 raft_consensus.cc:348] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.626057 28429 raft_consensus.cc:374] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.626094 28429 raft_consensus.cc:729] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.626199 28429 consensus_queue.cc:260] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.626488 28429 ts_tablet_manager.cc:1398] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.626639 28429 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:40.626771 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 (DEFAULT_TABLE table=rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:40.626977 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1417ea39bb2b4716bfb3d11c05853c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.627439 28429 tablet_bootstrap.cc:654] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.627532 28428 tablet_bootstrap.cc:492] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:40.628340 28428 tablet_bootstrap.cc:654] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.628386 28432 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:40.629190 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1417ea39bb2b4716bfb3d11c05853c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:40.629317 28432 tablet_bootstrap.cc:654] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.630808 28429 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:40.630879 28429 ts_tablet_manager.cc:1367] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:40.631001 28432 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:40.631078 28432 ts_tablet_manager.cc:1367] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:40.631269 28429 raft_consensus.cc:348] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.631366 28429 raft_consensus.cc:374] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.631402 28429 raft_consensus.cc:729] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.631500 28429 consensus_queue.cc:260] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.631479 28432 raft_consensus.cc:348] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.631585 28432 raft_consensus.cc:374] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.631606 28428 tablet_bootstrap.cc:492] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:40.631626 28432 raft_consensus.cc:729] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.631673 28428 ts_tablet_manager.cc:1367] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:40.631760 28429 ts_tablet_manager.cc:1398] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:40.631728 28432 consensus_queue.cc:260] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.631911 28429 tablet_bootstrap.cc:492] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:40.631896 28436 tablet_bootstrap.cc:492] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:40.631990 28432 ts_tablet_manager.cc:1398] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:40.632046 28428 raft_consensus.cc:348] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.632140 28428 raft_consensus.cc:374] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.632174 28428 raft_consensus.cc:729] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.632277 28428 consensus_queue.cc:260] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.632540 28428 ts_tablet_manager.cc:1398] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.632910 28436 tablet_bootstrap.cc:654] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.633142 28429 tablet_bootstrap.cc:654] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.634794 28429 tablet_bootstrap.cc:492] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:40.634888 28429 ts_tablet_manager.cc:1367] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:40.635409 28429 raft_consensus.cc:348] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.635491 28429 raft_consensus.cc:374] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.635509 28436 tablet_bootstrap.cc:492] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:40.635519 28429 raft_consensus.cc:729] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.635627 28429 consensus_queue.cc:260] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.635645 28436 ts_tablet_manager.cc:1367] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:40.635885 28429 ts_tablet_manager.cc:1398] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.636255 28436 raft_consensus.cc:348] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.636359 28436 raft_consensus.cc:374] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.636384 28436 raft_consensus.cc:729] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.636498 28436 consensus_queue.cc:260] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.636785 28436 ts_tablet_manager.cc:1398] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:40.637048 28436 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:40.637867 28436 tablet_bootstrap.cc:654] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:40.639906 28436 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:40.639971 28436 ts_tablet_manager.cc:1367] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:40.640290 28436 raft_consensus.cc:348] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.640367 28436 raft_consensus.cc:374] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:40.640391 28436 raft_consensus.cc:729] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:40.640472 28436 consensus_queue.cc:260] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.640687 28436 ts_tablet_manager.cc:1398] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:40.692224 28312 raft_consensus.cc:482] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:40.692322 28312 raft_consensus.cc:504] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.692683 28312 leader_election.cc:288] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:40.692945 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adcab4321d549919938465f639c9ddc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:40.692992 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adcab4321d549919938465f639c9ddc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:40.693076 26793 raft_consensus.cc:2454] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:40.693089 27063 raft_consensus.cc:2454] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:40.693329 26595 leader_election.cc:302] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:40.693503 28312 raft_consensus.cc:2781] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:40.693558 28312 raft_consensus.cc:482] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:40.693595 28312 raft_consensus.cc:3037] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.694353 28312 raft_consensus.cc:504] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.694635 28312 leader_election.cc:288] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:40.694924 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adcab4321d549919938465f639c9ddc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:40.694924 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adcab4321d549919938465f639c9ddc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:40.695030 26793 raft_consensus.cc:3037] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.695030 27063 raft_consensus.cc:3037] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.695873 27063 raft_consensus.cc:2454] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:40.695883 26793 raft_consensus.cc:2454] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:40.696115 26595 leader_election.cc:302] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:40.696270 28312 raft_consensus.cc:2781] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:40.696349 28312 raft_consensus.cc:686] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:40.696465 28312 consensus_queue.cc:237] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:40.698053 26512 catalog_manager.cc:5484] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:40.748327 28351 raft_consensus.cc:482] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:40.748430 28351 raft_consensus.cc:504] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.748798 28351 leader_election.cc:288] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:40.749078 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1417ea39bb2b4716bfb3d11c05853c03" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:40.749083 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1417ea39bb2b4716bfb3d11c05853c03" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:40.749213 26930 raft_consensus.cc:2454] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:40.749214 26659 raft_consensus.cc:2454] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:40.749466 26998 leader_election.cc:302] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:40.749629 28351 raft_consensus.cc:2781] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:40.749688 28351 raft_consensus.cc:482] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:40.749723 28351 raft_consensus.cc:3037] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.750522 28351 raft_consensus.cc:504] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.750816 28351 leader_election.cc:288] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:40.751078 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1417ea39bb2b4716bfb3d11c05853c03" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:40.751086 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1417ea39bb2b4716bfb3d11c05853c03" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:40.751171 26659 raft_consensus.cc:3037] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.751174 26930 raft_consensus.cc:3037] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.751842 26930 raft_consensus.cc:2454] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:40.751859 26659 raft_consensus.cc:2454] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:40.752120 26997 leader_election.cc:302] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:40.752262 28351 raft_consensus.cc:2781] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:40.752343 28351 raft_consensus.cc:686] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:40.752465 28351 consensus_queue.cc:237] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:40.754015 26512 catalog_manager.cc:5484] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:40.788720 28312 raft_consensus.cc:482] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:40.788838 28312 raft_consensus.cc:504] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.789187 28312 leader_election.cc:288] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:40.789585 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1f07f16b0f49b4b3c3cc5511604532" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:40.789623 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1f07f16b0f49b4b3c3cc5511604532" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:40.789710 26930 raft_consensus.cc:2454] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:40.789762 26793 raft_consensus.cc:2454] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:40.790007 26595 leader_election.cc:302] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:40.790200 28312 raft_consensus.cc:2781] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:40.790256 28312 raft_consensus.cc:482] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:40.790300 28312 raft_consensus.cc:3037] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.791061 28312 raft_consensus.cc:504] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.791352 28312 leader_election.cc:288] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:40.791651 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1f07f16b0f49b4b3c3cc5511604532" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:40.791661 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1f07f16b0f49b4b3c3cc5511604532" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:40.791760 26793 raft_consensus.cc:3037] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.791757 26930 raft_consensus.cc:3037] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:40.792456 26930 raft_consensus.cc:2454] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:40.792456 26793 raft_consensus.cc:2454] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:40.792734 26594 leader_election.cc:302] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:40.792887 28312 raft_consensus.cc:2781] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:40.792969 28312 raft_consensus.cc:686] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:40.793098 28312 consensus_queue.cc:237] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:40.794809 26512 catalog_manager.cc:5484] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:58:40.915386 28444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:40.915580 28444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:40.918900 28444 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_12'
I20240502 01:58:41.001518 26793 raft_consensus.cc:1261] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.001578 26930 raft_consensus.cc:1261] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.002166 28242 consensus_queue.cc:1035] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.002364 28312 consensus_queue.cc:1035] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.006603 26928 raft_consensus.cc:1261] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.006999 26660 raft_consensus.cc:1261] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.007916 27063 raft_consensus.cc:1261] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.008122 26795 raft_consensus.cc:1261] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.008106 28351 consensus_queue.cc:1035] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.008284 28242 consensus_queue.cc:1035] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.008955 28102 consensus_queue.cc:1035] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.010196 28312 consensus_queue.cc:1035] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.982 ms
time per row: 0.056991 ms
I20240502 01:58:41.070392 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:41.075520 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 039270108deb4f55b8458ceccf31daa6 (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.075728 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 039270108deb4f55b8458ceccf31daa6 (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.075811 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039270108deb4f55b8458ceccf31daa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.075979 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039270108deb4f55b8458ceccf31daa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.076087 26908 tablet_service.cc:1467] Processing CreateTablet for tablet e1950ace91824415b9bf3d1657bad72f (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.076294 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1950ace91824415b9bf3d1657bad72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.076282 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 039270108deb4f55b8458ceccf31daa6 (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.076501 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039270108deb4f55b8458ceccf31daa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.077908 27043 tablet_service.cc:1467] Processing CreateTablet for tablet e1950ace91824415b9bf3d1657bad72f (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.078152 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1950ace91824415b9bf3d1657bad72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.078516 28432 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:41.078891 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 052566382186437e8e0245de9f60c30c (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.079102 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052566382186437e8e0245de9f60c30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.079372 28432 tablet_bootstrap.cc:654] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.080456 28432 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:41.080540 28432 ts_tablet_manager.cc:1367] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:41.080827 28436 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:41.080929 28432 raft_consensus.cc:348] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.081023 28432 raft_consensus.cc:374] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.081053 28432 raft_consensus.cc:729] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.081166 28432 consensus_queue.cc:260] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.081449 28432 ts_tablet_manager.cc:1398] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:41.081486 28436 tablet_bootstrap.cc:654] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.081811 28432 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:41.081868 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 052566382186437e8e0245de9f60c30c (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.081924 28429 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:41.082067 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052566382186437e8e0245de9f60c30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.082615 28429 tablet_bootstrap.cc:654] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.082648 28432 tablet_bootstrap.cc:654] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.084041 26774 tablet_service.cc:1467] Processing CreateTablet for tablet e1950ace91824415b9bf3d1657bad72f (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.084050 26773 tablet_service.cc:1467] Processing CreateTablet for tablet 052566382186437e8e0245de9f60c30c (DEFAULT_TABLE table=rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.084277 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1950ace91824415b9bf3d1657bad72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.084450 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052566382186437e8e0245de9f60c30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.084841 28429 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:41.084911 28429 ts_tablet_manager.cc:1367] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:41.085314 28429 raft_consensus.cc:348] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.085409 28429 raft_consensus.cc:374] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.085440 28429 raft_consensus.cc:729] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.085547 28429 consensus_queue.cc:260] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.085822 28429 ts_tablet_manager.cc:1398] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.086766 28436 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:41.086836 28428 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:41.086846 28436 ts_tablet_manager.cc:1367] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:41.087301 28436 raft_consensus.cc:348] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.087404 28436 raft_consensus.cc:374] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.087447 28436 raft_consensus.cc:729] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.087572 28428 tablet_bootstrap.cc:654] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.087563 28436 consensus_queue.cc:260] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.087836 28436 ts_tablet_manager.cc:1398] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.087991 28436 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:41.088789 28436 tablet_bootstrap.cc:654] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.089010 28428 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:41.089066 28428 ts_tablet_manager.cc:1367] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:41.089392 28428 raft_consensus.cc:348] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.089481 28428 raft_consensus.cc:374] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.089505 28428 raft_consensus.cc:729] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.089601 28428 consensus_queue.cc:260] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.089857 28428 ts_tablet_manager.cc:1398] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.090780 28428 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:41.091601 28428 tablet_bootstrap.cc:654] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.092353 28436 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:41.092415 28436 ts_tablet_manager.cc:1367] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:41.092563 28428 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:41.092643 28428 ts_tablet_manager.cc:1367] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:41.092923 28436 raft_consensus.cc:348] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.093022 28436 raft_consensus.cc:374] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.093053 28436 raft_consensus.cc:729] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.093021 28428 raft_consensus.cc:348] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.093106 28428 raft_consensus.cc:374] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.093184 28428 raft_consensus.cc:729] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.093166 28436 consensus_queue.cc:260] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.093308 28428 consensus_queue.cc:260] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.093464 28436 ts_tablet_manager.cc:1398] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.093561 28428 ts_tablet_manager.cc:1398] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.094193 28428 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:41.094208 28432 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:41.094285 28432 ts_tablet_manager.cc:1367] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:58:41.094684 28432 raft_consensus.cc:348] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.094790 28432 raft_consensus.cc:374] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.094828 28432 raft_consensus.cc:729] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.094878 28428 tablet_bootstrap.cc:654] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.094944 28432 consensus_queue.cc:260] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.095211 28432 ts_tablet_manager.cc:1398] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.095369 28432 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:41.096154 28432 tablet_bootstrap.cc:654] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.097208 28432 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:41.097357 28432 ts_tablet_manager.cc:1367] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:41.097997 28432 raft_consensus.cc:348] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.098160 28428 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:41.098183 28432 raft_consensus.cc:374] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.098276 28432 raft_consensus.cc:729] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.098412 28432 consensus_queue.cc:260] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.098656 28432 ts_tablet_manager.cc:1398] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.098794 28428 ts_tablet_manager.cc:1367] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:41.099188 28428 raft_consensus.cc:348] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.099280 28428 raft_consensus.cc:374] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.099313 28428 raft_consensus.cc:729] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.099447 28428 consensus_queue.cc:260] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.099684 28428 ts_tablet_manager.cc:1398] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:41.113615 28475 raft_consensus.cc:482] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:41.113698 28475 raft_consensus.cc:504] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.114112 28475 leader_election.cc:288] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:41.114334 26928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1950ace91824415b9bf3d1657bad72f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:41.114408 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1950ace91824415b9bf3d1657bad72f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:41.114450 26928 raft_consensus.cc:2454] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:41.114511 26793 raft_consensus.cc:2454] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:41.114724 26998 leader_election.cc:302] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:41.114885 28475 raft_consensus.cc:2781] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:41.114943 28475 raft_consensus.cc:482] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:41.114979 28475 raft_consensus.cc:3037] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.115617 28475 raft_consensus.cc:504] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.115922 28475 leader_election.cc:288] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:41.116221 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1950ace91824415b9bf3d1657bad72f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:41.116307 26793 raft_consensus.cc:3037] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.116874 26793 raft_consensus.cc:2454] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:41.117058 26928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1950ace91824415b9bf3d1657bad72f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:41.117100 26997 leader_election.cc:302] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:41.117143 26928 raft_consensus.cc:3037] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.117270 28475 raft_consensus.cc:2781] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:41.117355 28475 raft_consensus.cc:686] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:41.117481 28475 consensus_queue.cc:237] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.118009 26928 raft_consensus.cc:2454] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:41.119084 26512 catalog_manager.cc:5484] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:41.228662 28422 raft_consensus.cc:482] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:41.228756 28422 raft_consensus.cc:504] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.229085 28422 leader_election.cc:288] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:41.229348 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039270108deb4f55b8458ceccf31daa6" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:41.229373 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039270108deb4f55b8458ceccf31daa6" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:41.229461 26793 raft_consensus.cc:2454] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:41.229489 27064 raft_consensus.cc:2454] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:41.229718 26861 leader_election.cc:302] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:41.229887 28422 raft_consensus.cc:2781] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:41.229948 28422 raft_consensus.cc:482] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:41.229993 28422 raft_consensus.cc:3037] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.230743 28422 raft_consensus.cc:504] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.231076 28422 leader_election.cc:288] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:41.231321 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039270108deb4f55b8458ceccf31daa6" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:41.231384 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039270108deb4f55b8458ceccf31daa6" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:41.231427 26793 raft_consensus.cc:3037] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.231509 27064 raft_consensus.cc:3037] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.232172 26793 raft_consensus.cc:2454] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:41.232188 27064 raft_consensus.cc:2454] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:41.232429 26861 leader_election.cc:302] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:41.232579 28422 raft_consensus.cc:2781] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:41.232724 28422 raft_consensus.cc:686] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:41.232872 28422 consensus_queue.cc:237] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:41.234572 26512 catalog_manager.cc:5484] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:41.389607 28297 raft_consensus.cc:482] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:41.389703 28297 raft_consensus.cc:504] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.390069 28297 leader_election.cc:288] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:41.390311 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052566382186437e8e0245de9f60c30c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:41.390379 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052566382186437e8e0245de9f60c30c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:41.390439 26660 raft_consensus.cc:2454] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:41.390507 27064 raft_consensus.cc:2454] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:41.390743 26729 leader_election.cc:302] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:41.390923 28297 raft_consensus.cc:2781] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:41.390988 28297 raft_consensus.cc:482] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:41.391027 28297 raft_consensus.cc:3037] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.391830 28297 raft_consensus.cc:504] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.392138 28297 leader_election.cc:288] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:41.392372 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052566382186437e8e0245de9f60c30c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:41.392410 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052566382186437e8e0245de9f60c30c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:41.392493 27064 raft_consensus.cc:3037] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.392488 26660 raft_consensus.cc:3037] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.393385 27064 raft_consensus.cc:2454] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:41.393385 26660 raft_consensus.cc:2454] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:41.393666 26730 leader_election.cc:302] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:41.393836 28297 raft_consensus.cc:2781] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:41.393914 28297 raft_consensus.cc:686] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:41.394207 28297 consensus_queue.cc:237] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.396008 26512 catalog_manager.cc:5484] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
W20240502 01:58:41.546370 28498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:41.546571 28498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:41.549607 28498 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 01:58:41.568162 28351 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_13'
I20240502 01:58:41.571094 28475 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.628769 27064 raft_consensus.cc:1261] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.628863 26793 raft_consensus.cc:1261] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.629232 28495 consensus_queue.cc:1035] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.629943 26660 raft_consensus.cc:1261] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.630373 28422 consensus_queue.cc:1035] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.630614 27064 raft_consensus.cc:1261] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:41.630977 28297 consensus_queue.cc:1035] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:41.633240 28105 consensus_queue.cc:1035] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 96.6402 ms
time per row: 0.0483201 ms
I20240502 01:58:41.690127 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:41.695051 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac02baf94743f1b6068748d83f0fe8 (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.695348 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac02baf94743f1b6068748d83f0fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.695557 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac02baf94743f1b6068748d83f0fe8 (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.695693 26909 tablet_service.cc:1467] Processing CreateTablet for tablet e60816eacfb547cda5a6c87c68bf101d (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.695819 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac02baf94743f1b6068748d83f0fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.695924 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 9bad5aeb22544e40b601ce3a216f89c7 (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.695978 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e60816eacfb547cda5a6c87c68bf101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.696115 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bad5aeb22544e40b601ce3a216f89c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.696249 27043 tablet_service.cc:1467] Processing CreateTablet for tablet e60816eacfb547cda5a6c87c68bf101d (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.696486 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e60816eacfb547cda5a6c87c68bf101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.698503 28562 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:41.698510 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac02baf94743f1b6068748d83f0fe8 (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.698731 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac02baf94743f1b6068748d83f0fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.699363 28562 tablet_bootstrap.cc:654] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.700364 28564 tablet_bootstrap.cc:492] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:41.700407 28562 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:41.700486 28562 ts_tablet_manager.cc:1367] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:41.700918 28562 raft_consensus.cc:348] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.701184 28564 tablet_bootstrap.cc:654] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.701159 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 9bad5aeb22544e40b601ce3a216f89c7 (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.701258 28562 raft_consensus.cc:374] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.701297 28562 raft_consensus.cc:729] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.701390 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bad5aeb22544e40b601ce3a216f89c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.701404 28562 consensus_queue.cc:260] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.701745 28562 ts_tablet_manager.cc:1398] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.701742 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 9bad5aeb22544e40b601ce3a216f89c7 (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.701943 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bad5aeb22544e40b601ce3a216f89c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.702857 28564 tablet_bootstrap.cc:492] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:41.702929 28564 ts_tablet_manager.cc:1367] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:41.703476 28564 raft_consensus.cc:348] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.703599 28564 raft_consensus.cc:374] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.703652 28564 raft_consensus.cc:729] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.703625 26641 tablet_service.cc:1467] Processing CreateTablet for tablet e60816eacfb547cda5a6c87c68bf101d (DEFAULT_TABLE table=rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:41.703788 28564 consensus_queue.cc:260] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.703871 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e60816eacfb547cda5a6c87c68bf101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:41.704160 28564 ts_tablet_manager.cc:1398] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.704350 28561 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:41.704435 28564 tablet_bootstrap.cc:492] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:41.705228 28564 tablet_bootstrap.cc:654] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.706259 28564 tablet_bootstrap.cc:492] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:41.706346 28564 ts_tablet_manager.cc:1367] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:41.706403 28560 tablet_bootstrap.cc:492] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:41.706686 28561 tablet_bootstrap.cc:654] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.706743 28564 raft_consensus.cc:348] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.706823 28564 raft_consensus.cc:374] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.706854 28564 raft_consensus.cc:729] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.706957 28564 consensus_queue.cc:260] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.707242 28564 ts_tablet_manager.cc:1398] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:41.707362 28560 tablet_bootstrap.cc:654] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.708371 28560 tablet_bootstrap.cc:492] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:41.708461 28560 ts_tablet_manager.cc:1367] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:41.708727 28561 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:41.708809 28561 ts_tablet_manager.cc:1367] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:41.708851 28560 raft_consensus.cc:348] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.708969 28560 raft_consensus.cc:374] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.709007 28560 raft_consensus.cc:729] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.709127 28560 consensus_queue.cc:260] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.709389 28560 ts_tablet_manager.cc:1398] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:41.709553 28560 tablet_bootstrap.cc:492] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:41.709534 28561 raft_consensus.cc:348] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.709628 28561 raft_consensus.cc:374] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.709661 28561 raft_consensus.cc:729] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.709760 28561 consensus_queue.cc:260] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.710127 28561 ts_tablet_manager.cc:1398] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.710261 28560 tablet_bootstrap.cc:654] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.710776 28561 tablet_bootstrap.cc:492] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:41.712144 28560 tablet_bootstrap.cc:492] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:41.712242 28560 ts_tablet_manager.cc:1367] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:41.712615 28560 raft_consensus.cc:348] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.712719 28560 raft_consensus.cc:374] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.712751 28560 raft_consensus.cc:729] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.712760 28561 tablet_bootstrap.cc:654] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.712872 28560 consensus_queue.cc:260] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.713228 28560 ts_tablet_manager.cc:1398] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.713376 28560 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:41.713826 28561 tablet_bootstrap.cc:492] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:41.713908 28561 ts_tablet_manager.cc:1367] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:41.714061 28560 tablet_bootstrap.cc:654] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.714330 28561 raft_consensus.cc:348] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.714430 28561 raft_consensus.cc:374] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.714463 28561 raft_consensus.cc:729] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.714650 28561 consensus_queue.cc:260] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.715004 28560 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:41.715035 28561 ts_tablet_manager.cc:1398] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.715090 28560 ts_tablet_manager.cc:1367] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:41.715277 28561 tablet_bootstrap.cc:492] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:41.715484 28560 raft_consensus.cc:348] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.715577 28560 raft_consensus.cc:374] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.715608 28560 raft_consensus.cc:729] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.715721 28560 consensus_queue.cc:260] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.715972 28560 ts_tablet_manager.cc:1398] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:41.716054 28561 tablet_bootstrap.cc:654] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:41.717058 28561 tablet_bootstrap.cc:492] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:41.717136 28561 ts_tablet_manager.cc:1367] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:41.717482 28561 raft_consensus.cc:348] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.717586 28561 raft_consensus.cc:374] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:41.717617 28561 raft_consensus.cc:729] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:41.717739 28561 consensus_queue.cc:260] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.718043 28561 ts_tablet_manager.cc:1398] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:41.778147 28547 raft_consensus.cc:482] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:41.778268 28547 raft_consensus.cc:504] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.778632 28547 leader_election.cc:288] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:41.778926 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60816eacfb547cda5a6c87c68bf101d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:41.779039 26659 raft_consensus.cc:2454] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:41.779027 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60816eacfb547cda5a6c87c68bf101d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:41.779166 26929 raft_consensus.cc:2454] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:41.779301 26997 leader_election.cc:302] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:41.779446 28547 raft_consensus.cc:2781] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:41.779515 28547 raft_consensus.cc:482] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:41.779572 28547 raft_consensus.cc:3037] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.780352 28547 raft_consensus.cc:504] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.780660 28547 leader_election.cc:288] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:41.780915 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60816eacfb547cda5a6c87c68bf101d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:41.780933 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60816eacfb547cda5a6c87c68bf101d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:41.781035 26929 raft_consensus.cc:3037] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.781035 26659 raft_consensus.cc:3037] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.781926 26659 raft_consensus.cc:2454] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:41.781926 26929 raft_consensus.cc:2454] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:41.782238 26997 leader_election.cc:302] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:41.782387 28547 raft_consensus.cc:2781] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:41.782469 28547 raft_consensus.cc:686] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:41.782604 28547 consensus_queue.cc:237] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:41.784318 26510 catalog_manager.cc:5484] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:41.933955 28483 raft_consensus.cc:482] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:41.934062 28483 raft_consensus.cc:504] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.934381 28483 leader_election.cc:288] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:41.934661 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad5aeb22544e40b601ce3a216f89c7" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:41.934661 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad5aeb22544e40b601ce3a216f89c7" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:41.934791 26793 raft_consensus.cc:2454] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:41.934799 26929 raft_consensus.cc:2454] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:41.935072 28422 raft_consensus.cc:482] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:41.935086 26595 leader_election.cc:302] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:41.935161 28422 raft_consensus.cc:504] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.935247 28483 raft_consensus.cc:2781] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:41.935292 28483 raft_consensus.cc:482] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:41.935333 28483 raft_consensus.cc:3037] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.935552 28422 leader_election.cc:288] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:41.935786 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac02baf94743f1b6068748d83f0fe8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:41.935866 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac02baf94743f1b6068748d83f0fe8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:41.935920 26659 raft_consensus.cc:2454] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:41.935968 26793 raft_consensus.cc:2454] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:41.936148 26861 leader_election.cc:302] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:41.936332 28422 raft_consensus.cc:2781] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:41.936334 28483 raft_consensus.cc:504] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.936395 28422 raft_consensus.cc:482] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:41.936441 28422 raft_consensus.cc:3037] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.936636 28483 leader_election.cc:288] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:41.936882 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad5aeb22544e40b601ce3a216f89c7" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:41.936916 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad5aeb22544e40b601ce3a216f89c7" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:41.936977 26929 raft_consensus.cc:3037] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.937008 26793 raft_consensus.cc:3037] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.937355 28422 raft_consensus.cc:504] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.937662 28422 leader_election.cc:288] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:41.937737 26793 raft_consensus.cc:2454] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:41.937927 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac02baf94743f1b6068748d83f0fe8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:41.937973 26594 leader_election.cc:302] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:41.938046 26659 raft_consensus.cc:3037] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.938019 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac02baf94743f1b6068748d83f0fe8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:41.938129 26793 raft_consensus.cc:3037] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:41.938124 28483 raft_consensus.cc:2781] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:41.938345 28483 raft_consensus.cc:686] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:41.938482 28483 consensus_queue.cc:237] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.938885 26659 raft_consensus.cc:2454] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:41.938921 26793 raft_consensus.cc:2454] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:41.939184 26861 leader_election.cc:302] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:41.939224 26929 raft_consensus.cc:2454] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:41.939457 28422 raft_consensus.cc:2781] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:41.939532 28422 raft_consensus.cc:686] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:41.939648 28422 consensus_queue.cc:237] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:41.940659 26510 catalog_manager.cc:5484] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:41.941471 26509 catalog_manager.cc:5484] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:58:42.091605 28577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:42.091789 28577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:42.094874 28577 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_14'
I20240502 01:58:42.186543 26659 raft_consensus.cc:1261] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.186928 28351 consensus_queue.cc:1035] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.187713 26929 raft_consensus.cc:1261] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.188110 28351 consensus_queue.cc:1035] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.189020 26660 raft_consensus.cc:1261] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.189350 26793 raft_consensus.cc:1261] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.189404 28422 consensus_queue.cc:1035] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.191041 28539 consensus_queue.cc:1035] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.192840 26794 raft_consensus.cc:1261] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.193289 28574 consensus_queue.cc:1035] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.193948 26928 raft_consensus.cc:1261] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.194643 28574 consensus_queue.cc:1035] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 121.86 ms
time per row: 0.0609298 ms
I20240502 01:58:42.254391 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:42.259366 26641 tablet_service.cc:1467] Processing CreateTablet for tablet e24e0bd4a1df4fed876e1c4c50af7d70 (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.259408 27043 tablet_service.cc:1467] Processing CreateTablet for tablet e24e0bd4a1df4fed876e1c4c50af7d70 (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.259651 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24e0bd4a1df4fed876e1c4c50af7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.259660 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24e0bd4a1df4fed876e1c4c50af7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.260286 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.260265 26774 tablet_service.cc:1467] Processing CreateTablet for tablet e24e0bd4a1df4fed876e1c4c50af7d70 (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.260551 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.260672 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24e0bd4a1df4fed876e1c4c50af7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.261969 26640 tablet_service.cc:1467] Processing CreateTablet for tablet a0d96b487a964b90b720d8977a96fc5b (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.262183 28638 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:42.262223 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d96b487a964b90b720d8977a96fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.262876 28640 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:42.262878 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.263027 28638 tablet_bootstrap.cc:654] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.263114 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.263665 28640 tablet_bootstrap.cc:654] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.264144 26908 tablet_service.cc:1467] Processing CreateTablet for tablet a0d96b487a964b90b720d8977a96fc5b (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.264364 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d96b487a964b90b720d8977a96fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.264530 28640 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:42.264587 28640 ts_tablet_manager.cc:1367] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:42.264909 28640 raft_consensus.cc:348] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.264979 28640 raft_consensus.cc:374] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.264997 28640 raft_consensus.cc:729] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.265126 28640 consensus_queue.cc:260] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.265242 28643 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:42.265467 28640 ts_tablet_manager.cc:1398] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:42.265614 28640 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:42.266216 28643 tablet_bootstrap.cc:654] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.266575 28640 tablet_bootstrap.cc:654] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.266647 28638 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:42.266729 28638 ts_tablet_manager.cc:1367] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:42.267093 28638 raft_consensus.cc:348] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.267207 28638 raft_consensus.cc:374] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.267241 28638 raft_consensus.cc:729] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.267340 28638 consensus_queue.cc:260] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.267354 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.267541 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.267606 28638 ts_tablet_manager.cc:1398] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:42.267875 27043 tablet_service.cc:1467] Processing CreateTablet for tablet a0d96b487a964b90b720d8977a96fc5b (DEFAULT_TABLE table=rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.268085 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d96b487a964b90b720d8977a96fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.269104 28638 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:42.269767 28639 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:42.270210 28638 tablet_bootstrap.cc:654] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.270578 28639 tablet_bootstrap.cc:654] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.271492 28639 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:42.271571 28639 ts_tablet_manager.cc:1367] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:42.271759 28640 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:42.271854 28640 ts_tablet_manager.cc:1367] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 01:58:42.271940 28639 raft_consensus.cc:348] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.272046 28639 raft_consensus.cc:374] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.272085 28639 raft_consensus.cc:729] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.272190 28639 consensus_queue.cc:260] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.272395 28640 raft_consensus.cc:348] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.272454 28639 ts_tablet_manager.cc:1398] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:42.272531 28640 raft_consensus.cc:374] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.272619 28640 raft_consensus.cc:729] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.272679 28639 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:42.272857 28643 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:42.272823 28640 consensus_queue.cc:260] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.272931 28643 ts_tablet_manager.cc:1367] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 01:58:42.273012 28638 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:42.273084 28638 ts_tablet_manager.cc:1367] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 01:58:42.273147 28640 ts_tablet_manager.cc:1398] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.273339 28643 raft_consensus.cc:348] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.273406 28643 raft_consensus.cc:374] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.273423 28643 raft_consensus.cc:729] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.273465 28639 tablet_bootstrap.cc:654] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.273463 28638 raft_consensus.cc:348] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.273515 28643 consensus_queue.cc:260] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.273562 28638 raft_consensus.cc:374] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.273619 28638 raft_consensus.cc:729] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.273751 28643 ts_tablet_manager.cc:1398] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.273728 28638 consensus_queue.cc:260] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.273878 28643 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:42.273988 28638 ts_tablet_manager.cc:1398] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.274240 28638 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:42.274734 28643 tablet_bootstrap.cc:654] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.275038 28638 tablet_bootstrap.cc:654] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.275281 28639 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:42.275368 28639 ts_tablet_manager.cc:1367] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:42.275748 28639 raft_consensus.cc:348] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.275848 28639 raft_consensus.cc:374] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.275878 28639 raft_consensus.cc:729] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.275938 28638 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:42.275974 28639 consensus_queue.cc:260] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.276011 28638 ts_tablet_manager.cc:1367] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:42.276208 28639 ts_tablet_manager.cc:1398] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:42.276418 28638 raft_consensus.cc:348] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.276508 28638 raft_consensus.cc:374] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.276535 28638 raft_consensus.cc:729] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.276649 28638 consensus_queue.cc:260] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.276888 28638 ts_tablet_manager.cc:1398] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:42.280887 28643 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:42.280952 28643 ts_tablet_manager.cc:1367] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:42.281242 28643 raft_consensus.cc:348] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.281328 28643 raft_consensus.cc:374] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.281365 28643 raft_consensus.cc:729] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.281471 28643 consensus_queue.cc:260] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.281708 28643 ts_tablet_manager.cc:1398] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:42.330837 28351 raft_consensus.cc:482] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:42.330924 28351 raft_consensus.cc:504] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.331238 28351 leader_election.cc:288] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:42.331499 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24e0bd4a1df4fed876e1c4c50af7d70" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:42.331540 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24e0bd4a1df4fed876e1c4c50af7d70" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:42.331604 26659 raft_consensus.cc:2454] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:42.331637 26794 raft_consensus.cc:2454] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:42.331854 26997 leader_election.cc:302] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:42.331996 28351 raft_consensus.cc:2781] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:42.332051 28351 raft_consensus.cc:482] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:42.332075 28351 raft_consensus.cc:3037] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.332784 28351 raft_consensus.cc:504] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.333055 28351 leader_election.cc:288] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:42.333323 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24e0bd4a1df4fed876e1c4c50af7d70" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:42.333423 26659 raft_consensus.cc:3037] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.333405 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24e0bd4a1df4fed876e1c4c50af7d70" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:42.333500 26794 raft_consensus.cc:3037] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.334313 26659 raft_consensus.cc:2454] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:42.334385 26794 raft_consensus.cc:2454] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:42.334553 26997 leader_election.cc:302] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:42.334689 28351 raft_consensus.cc:2781] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:42.334903 28351 raft_consensus.cc:686] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:42.335031 28351 consensus_queue.cc:237] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.336525 26515 catalog_manager.cc:5484] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:42.524231 28105 raft_consensus.cc:482] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:42.524334 28105 raft_consensus.cc:504] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.524674 28105 leader_election.cc:288] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:42.524932 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2aaf8eaa844c0390909ccfc0b4bb09" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:42.524991 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2aaf8eaa844c0390909ccfc0b4bb09" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:42.525059 27063 raft_consensus.cc:2454] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:42.525106 26930 raft_consensus.cc:2454] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:42.525354 26730 leader_election.cc:302] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:42.525511 28105 raft_consensus.cc:2781] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:42.525570 28105 raft_consensus.cc:482] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:42.525616 28105 raft_consensus.cc:3037] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.526638 28105 raft_consensus.cc:504] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.526973 28105 leader_election.cc:288] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:42.527266 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2aaf8eaa844c0390909ccfc0b4bb09" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:42.527402 27063 raft_consensus.cc:3037] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.528199 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b2aaf8eaa844c0390909ccfc0b4bb09" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:42.528362 27063 raft_consensus.cc:2454] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:42.528395 26930 raft_consensus.cc:3037] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.528723 26730 leader_election.cc:302] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:42.528892 28105 raft_consensus.cc:2781] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:42.529068 28105 raft_consensus.cc:686] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:42.529356 28105 consensus_queue.cc:237] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:42.529462 26930 raft_consensus.cc:2454] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:42.531526 26515 catalog_manager.cc:5484] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:42.556950 28351 raft_consensus.cc:482] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:42.557088 28351 raft_consensus.cc:504] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.557489 28351 leader_election.cc:288] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:42.557833 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d96b487a964b90b720d8977a96fc5b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:42.557847 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d96b487a964b90b720d8977a96fc5b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:42.557960 26930 raft_consensus.cc:2454] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:42.557972 26659 raft_consensus.cc:2454] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:42.558293 26997 leader_election.cc:302] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:42.558455 28351 raft_consensus.cc:2781] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:42.558514 28351 raft_consensus.cc:482] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:42.558562 28351 raft_consensus.cc:3037] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.559517 28351 raft_consensus.cc:504] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.559885 28351 leader_election.cc:288] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:42.560160 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d96b487a964b90b720d8977a96fc5b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:42.560212 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d96b487a964b90b720d8977a96fc5b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:42.560273 26659 raft_consensus.cc:3037] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.560310 26930 raft_consensus.cc:3037] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.561127 26659 raft_consensus.cc:2454] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:42.561139 26930 raft_consensus.cc:2454] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:42.561419 26998 leader_election.cc:302] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:42.561627 28351 raft_consensus.cc:2781] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:42.561712 28351 raft_consensus.cc:686] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:42.561832 28351 consensus_queue.cc:237] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.563750 26515 catalog_manager.cc:5484] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
W20240502 01:58:42.732266 28656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:42.732456 28656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:42.735698 28656 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_15'
I20240502 01:58:42.827898 26794 raft_consensus.cc:1261] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.828248 26659 raft_consensus.cc:1261] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.828972 28351 consensus_queue.cc:1035] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.829272 28547 consensus_queue.cc:1035] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.832103 26659 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.832201 26930 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.832593 28351 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.832705 28547 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.835870 27063 raft_consensus.cc:1261] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.836169 26928 raft_consensus.cc:1261] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:42.836524 28105 consensus_queue.cc:1035] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.836791 28105 consensus_queue.cc:1035] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:42.853791 28694 mvcc.cc:205] Tried to move back new op lower bound from 7023063543116292096 to 7023063542014681088. Current Snapshot: MvccSnapshot[applied={T|T < 7023063543116292096}]
I20240502 01:58:42.855592 28685 mvcc.cc:205] Tried to move back new op lower bound from 7023063543116292096 to 7023063542014681088. Current Snapshot: MvccSnapshot[applied={T|T < 7023063543116292096}]
INSERT report
rows total: 2000
time total: 107.366 ms
time per row: 0.0536831 ms
I20240502 01:58:42.887905 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:42.893491 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 507cc922d2884f48b0f4c43185b4225d (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.893491 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 507cc922d2884f48b0f4c43185b4225d (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.893651 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.893807 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507cc922d2884f48b0f4c43185b4225d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.893813 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507cc922d2884f48b0f4c43185b4225d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.894373 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 7efa5c32e6914d818262d473d71599a8 (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.894443 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.894582 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efa5c32e6914d818262d473d71599a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.894644 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.896332 28717 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:42.896546 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.896818 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 507cc922d2884f48b0f4c43185b4225d (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.896874 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 7efa5c32e6914d818262d473d71599a8 (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.897095 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507cc922d2884f48b0f4c43185b4225d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.897239 28718 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:42.897231 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.897472 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.898301 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efa5c32e6914d818262d473d71599a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.898429 28718 tablet_bootstrap.cc:654] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.898617 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 7efa5c32e6914d818262d473d71599a8 (DEFAULT_TABLE table=rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:42.898849 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efa5c32e6914d818262d473d71599a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:42.899430 28720 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:42.900086 28720 tablet_bootstrap.cc:654] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.901078 28722 tablet_bootstrap.cc:492] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:42.901094 28720 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:42.901167 28720 ts_tablet_manager.cc:1367] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:42.901660 28720 raft_consensus.cc:348] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:42.901868 28720 raft_consensus.cc:374] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.901932 28722 tablet_bootstrap.cc:654] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.901950 28720 raft_consensus.cc:729] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.902127 28720 consensus_queue.cc:260] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:42.902506 28720 ts_tablet_manager.cc:1398] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.902714 28720 tablet_bootstrap.cc:492] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:42.903584 28720 tablet_bootstrap.cc:654] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.904049 28717 tablet_bootstrap.cc:654] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.904340 28718 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:42.904430 28718 ts_tablet_manager.cc:1367] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 01:58:42.904814 28718 raft_consensus.cc:348] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:42.904907 28718 raft_consensus.cc:374] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.904937 28718 raft_consensus.cc:729] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.905076 28718 consensus_queue.cc:260] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:42.905185 28722 tablet_bootstrap.cc:492] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:42.905184 28717 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:42.905268 28722 ts_tablet_manager.cc:1367] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:42.905273 28717 ts_tablet_manager.cc:1367] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:42.905357 28718 ts_tablet_manager.cc:1398] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.905498 28718 tablet_bootstrap.cc:492] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:42.905689 28722 raft_consensus.cc:348] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.905694 28717 raft_consensus.cc:348] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:42.905767 28722 raft_consensus.cc:374] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.905787 28722 raft_consensus.cc:729] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.905790 28717 raft_consensus.cc:374] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.905819 28717 raft_consensus.cc:729] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.905882 28722 consensus_queue.cc:260] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.905928 28717 consensus_queue.cc:260] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:42.906162 28722 ts_tablet_manager.cc:1398] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:42.906208 28717 ts_tablet_manager.cc:1398] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.906360 28718 tablet_bootstrap.cc:654] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.906386 28717 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:42.906821 28722 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:42.907181 28717 tablet_bootstrap.cc:654] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.907632 28722 tablet_bootstrap.cc:654] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.909467 28718 tablet_bootstrap.cc:492] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:42.909552 28718 ts_tablet_manager.cc:1367] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:42.909993 28718 raft_consensus.cc:348] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.910125 28718 raft_consensus.cc:374] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.910540 28718 raft_consensus.cc:729] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.910712 28718 consensus_queue.cc:260] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.911028 28718 ts_tablet_manager.cc:1398] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:42.911201 28718 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:42.912395 28718 tablet_bootstrap.cc:654] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:42.913309 28718 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:42.913383 28718 ts_tablet_manager.cc:1367] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:42.913396 28717 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:42.913470 28717 ts_tablet_manager.cc:1367] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240502 01:58:42.913792 28718 raft_consensus.cc:348] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.913883 28718 raft_consensus.cc:374] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.913910 28718 raft_consensus.cc:729] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.913873 28717 raft_consensus.cc:348] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.913967 28717 raft_consensus.cc:374] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.913997 28717 raft_consensus.cc:729] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.914011 28718 consensus_queue.cc:260] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.914129 28717 consensus_queue.cc:260] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.914307 28718 ts_tablet_manager.cc:1398] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.914497 28720 tablet_bootstrap.cc:492] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:42.914592 28720 ts_tablet_manager.cc:1367] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240502 01:58:42.914990 28717 ts_tablet_manager.cc:1398] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:42.915041 28720 raft_consensus.cc:348] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.915163 28720 raft_consensus.cc:374] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.915196 28720 raft_consensus.cc:729] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.915300 28720 consensus_queue.cc:260] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.915560 28720 ts_tablet_manager.cc:1398] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:42.915975 28722 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:42.916218 28722 ts_tablet_manager.cc:1367] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:42.916584 28722 raft_consensus.cc:348] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.916733 28722 raft_consensus.cc:374] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:42.916781 28722 raft_consensus.cc:729] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:42.916935 28722 consensus_queue.cc:260] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.917250 28722 ts_tablet_manager.cc:1398] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:42.944736 28105 raft_consensus.cc:482] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:42.944842 28105 raft_consensus.cc:504] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.945215 28105 leader_election.cc:288] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:42.945492 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2ebe00a52240cb9ac5fc8fea9d7151" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:42.945516 26928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2ebe00a52240cb9ac5fc8fea9d7151" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:42.945632 26928 raft_consensus.cc:2454] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:42.945636 26660 raft_consensus.cc:2454] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:42.945868 26729 leader_election.cc:302] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:42.946041 28105 raft_consensus.cc:2781] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:42.946098 28105 raft_consensus.cc:482] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:42.946132 28105 raft_consensus.cc:3037] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.946864 28105 raft_consensus.cc:504] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.947166 28105 leader_election.cc:288] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:42.947427 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2ebe00a52240cb9ac5fc8fea9d7151" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:42.947441 26928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2ebe00a52240cb9ac5fc8fea9d7151" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:42.947525 26928 raft_consensus.cc:3037] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.947527 26660 raft_consensus.cc:3037] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.948222 26660 raft_consensus.cc:2454] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:42.948244 26928 raft_consensus.cc:2454] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:42.948491 26729 leader_election.cc:302] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:42.948684 28105 raft_consensus.cc:2781] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:42.948753 28105 raft_consensus.cc:686] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:42.948873 28105 consensus_queue.cc:237] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:42.950661 26512 catalog_manager.cc:5484] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:42.963511 28422 raft_consensus.cc:482] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:42.963646 28422 raft_consensus.cc:504] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.964038 28422 leader_election.cc:288] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:42.964423 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa5c32e6914d818262d473d71599a8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:42.964509 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa5c32e6914d818262d473d71599a8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:42.964583 26660 raft_consensus.cc:2454] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:42.964628 27063 raft_consensus.cc:2454] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:42.964891 26861 leader_election.cc:302] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:42.965057 28422 raft_consensus.cc:2781] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:42.965119 28422 raft_consensus.cc:482] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:42.965160 28422 raft_consensus.cc:3037] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.966208 28422 raft_consensus.cc:504] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.966552 28422 leader_election.cc:288] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:42.966955 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa5c32e6914d818262d473d71599a8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:42.967064 26660 raft_consensus.cc:3037] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.967046 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efa5c32e6914d818262d473d71599a8" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:42.967191 27063 raft_consensus.cc:3037] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:42.967983 26660 raft_consensus.cc:2454] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:42.968118 27063 raft_consensus.cc:2454] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:42.968230 26861 leader_election.cc:302] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:42.968499 28422 raft_consensus.cc:2781] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:42.968708 28422 raft_consensus.cc:686] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:42.968843 28422 consensus_queue.cc:237] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:42.970726 26512 catalog_manager.cc:5484] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:43.235373 28105 raft_consensus.cc:482] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:43.235541 28105 raft_consensus.cc:504] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.235953 28105 leader_election.cc:288] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:43.236305 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507cc922d2884f48b0f4c43185b4225d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:43.236353 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507cc922d2884f48b0f4c43185b4225d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:43.236454 27063 raft_consensus.cc:2454] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:43.236454 26660 raft_consensus.cc:2454] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:43.236701 26730 leader_election.cc:302] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.236871 28105 raft_consensus.cc:2781] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:43.236936 28105 raft_consensus.cc:482] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:43.236961 28105 raft_consensus.cc:3037] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.237741 28105 raft_consensus.cc:504] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.238097 28105 leader_election.cc:288] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:43.238454 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507cc922d2884f48b0f4c43185b4225d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:43.238574 26659 raft_consensus.cc:3037] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.239326 26659 raft_consensus.cc:2454] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:43.239544 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507cc922d2884f48b0f4c43185b4225d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:43.239604 26729 leader_election.cc:302] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.239647 27063 raft_consensus.cc:3037] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.239753 28653 raft_consensus.cc:2781] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:43.239845 28653 raft_consensus.cc:686] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:43.239974 28653 consensus_queue.cc:237] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.241447 27063 raft_consensus.cc:2454] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:43.241986 26512 catalog_manager.cc:5484] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
W20240502 01:58:43.369251 28735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:43.369451 28735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:43.372388 28735 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 01:58:43.373591 28422 consensus_queue.cc:1035] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:43.377884 28422 consensus_queue.cc:1035] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_16'
I20240502 01:58:43.418242 28105 consensus_queue.cc:1035] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:43.422693 28733 consensus_queue.cc:1035] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:43.464542 26660 raft_consensus.cc:1261] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:43.465230 28653 consensus_queue.cc:1035] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:43.471118 28773 mvcc.cc:205] Tried to move back new op lower bound from 7023063545705447424 to 7023063544792109056. Current Snapshot: MvccSnapshot[applied={T|T < 7023063545705447424}]
I20240502 01:58:43.471475 27063 raft_consensus.cc:1261] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:43.471899 28733 consensus_queue.cc:1035] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 92.2224 ms
time per row: 0.0461112 ms
I20240502 01:58:43.512950 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:43.520119 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.520233 27044 tablet_service.cc:1467] Processing CreateTablet for tablet b265eec55ee1406b9ff99c159304fa65 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.520315 26641 tablet_service.cc:1467] Processing CreateTablet for tablet c187cf1abe124e73badcb06ca1a489c4 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.520428 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.520442 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b265eec55ee1406b9ff99c159304fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.520599 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187cf1abe124e73badcb06ca1a489c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.522743 26908 tablet_service.cc:1467] Processing CreateTablet for tablet b265eec55ee1406b9ff99c159304fa65 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.522974 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b265eec55ee1406b9ff99c159304fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.523391 28796 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:43.523470 27043 tablet_service.cc:1467] Processing CreateTablet for tablet c187cf1abe124e73badcb06ca1a489c4 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.523669 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187cf1abe124e73badcb06ca1a489c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.524267 28796 tablet_bootstrap.cc:654] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.525295 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.525439 26773 tablet_service.cc:1467] Processing CreateTablet for tablet c187cf1abe124e73badcb06ca1a489c4 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.525527 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.525660 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187cf1abe124e73badcb06ca1a489c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.525255 26775 tablet_service.cc:1467] Processing CreateTablet for tablet b265eec55ee1406b9ff99c159304fa65 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.525925 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b265eec55ee1406b9ff99c159304fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.526285 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 (DEFAULT_TABLE table=rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:43.526495 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:43.527724 28795 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:43.527882 28794 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:43.528601 28795 tablet_bootstrap.cc:654] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.528641 28798 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:43.528944 28794 tablet_bootstrap.cc:654] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.529429 28798 tablet_bootstrap.cc:654] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.529682 28795 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:43.529760 28795 ts_tablet_manager.cc:1367] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:43.529870 28794 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:43.529965 28794 ts_tablet_manager.cc:1367] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:43.530215 28795 raft_consensus.cc:348] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.530321 28795 raft_consensus.cc:374] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.530350 28795 raft_consensus.cc:729] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.530432 28794 raft_consensus.cc:348] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.530458 28795 consensus_queue.cc:260] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.530516 28798 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:43.530525 28794 raft_consensus.cc:374] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.530531 28796 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:43.530555 28794 raft_consensus.cc:729] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.530584 28798 ts_tablet_manager.cc:1367] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:43.530599 28796 ts_tablet_manager.cc:1367] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:58:43.530673 28794 consensus_queue.cc:260] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.530726 28795 ts_tablet_manager.cc:1398] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:43.530946 28794 ts_tablet_manager.cc:1398] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:43.531018 28796 raft_consensus.cc:348] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.531086 28794 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:43.531109 28796 raft_consensus.cc:374] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.531136 28796 raft_consensus.cc:729] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.531239 28796 consensus_queue.cc:260] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.531493 28796 ts_tablet_manager.cc:1398] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:43.531574 28795 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:43.531898 28794 tablet_bootstrap.cc:654] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.532366 28795 tablet_bootstrap.cc:654] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.533461 28798 raft_consensus.cc:348] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.533630 28798 raft_consensus.cc:374] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.533776 28798 raft_consensus.cc:729] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.533977 28798 consensus_queue.cc:260] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.534144 28795 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:43.534226 28795 ts_tablet_manager.cc:1367] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:43.534451 28798 ts_tablet_manager.cc:1398] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:43.534564 28794 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:43.534657 28798 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:43.534633 28795 raft_consensus.cc:348] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.534731 28795 raft_consensus.cc:374] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.534768 28795 raft_consensus.cc:729] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.534839 28794 ts_tablet_manager.cc:1367] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:43.534883 28795 consensus_queue.cc:260] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.535144 28795 ts_tablet_manager.cc:1398] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:43.535311 28795 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:43.535604 28798 tablet_bootstrap.cc:654] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.535907 28794 raft_consensus.cc:348] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.536062 28794 raft_consensus.cc:374] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.536082 28795 tablet_bootstrap.cc:654] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.536149 28794 raft_consensus.cc:729] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.536345 28794 consensus_queue.cc:260] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.536760 28794 ts_tablet_manager.cc:1398] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:43.537417 28795 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:43.537417 28798 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:43.537492 28798 ts_tablet_manager.cc:1367] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:43.537492 28795 ts_tablet_manager.cc:1367] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:43.537906 28795 raft_consensus.cc:348] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.537923 28798 raft_consensus.cc:348] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.538007 28795 raft_consensus.cc:374] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.538059 28795 raft_consensus.cc:729] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.538079 28798 raft_consensus.cc:374] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.538174 28798 raft_consensus.cc:729] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.538172 28795 consensus_queue.cc:260] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.538319 28798 consensus_queue.cc:260] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.538436 28795 ts_tablet_manager.cc:1398] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:43.538646 28798 ts_tablet_manager.cc:1398] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:43.538815 28798 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:43.539824 28798 tablet_bootstrap.cc:654] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:43.541357 28798 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:43.541426 28798 ts_tablet_manager.cc:1367] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:43.541821 28798 raft_consensus.cc:348] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.541898 28798 raft_consensus.cc:374] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:43.541924 28798 raft_consensus.cc:729] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:43.542068 28798 consensus_queue.cc:260] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.542335 28798 ts_tablet_manager.cc:1398] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:43.621383 28351 raft_consensus.cc:482] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:43.621523 28351 raft_consensus.cc:504] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.621894 28351 leader_election.cc:288] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:43.622278 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187cf1abe124e73badcb06ca1a489c4" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:43.622314 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187cf1abe124e73badcb06ca1a489c4" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:43.622417 26660 raft_consensus.cc:2454] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:43.622447 26794 raft_consensus.cc:2454] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:43.622736 26997 leader_election.cc:302] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.622941 28351 raft_consensus.cc:2781] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:43.622999 28351 raft_consensus.cc:482] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:43.623042 28351 raft_consensus.cc:3037] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.623826 28351 raft_consensus.cc:504] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.624115 28351 leader_election.cc:288] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:43.624404 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187cf1abe124e73badcb06ca1a489c4" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:43.624454 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c187cf1abe124e73badcb06ca1a489c4" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:43.624507 26660 raft_consensus.cc:3037] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.624564 26794 raft_consensus.cc:3037] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.625255 26660 raft_consensus.cc:2454] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:43.625272 26794 raft_consensus.cc:2454] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:43.625527 26997 leader_election.cc:302] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.625779 28351 raft_consensus.cc:2781] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:43.625876 28351 raft_consensus.cc:686] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:43.625999 28351 consensus_queue.cc:237] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:43.627893 26509 catalog_manager.cc:5484] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:43.769778 28789 raft_consensus.cc:482] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:43.769891 28789 raft_consensus.cc:504] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.770289 28789 leader_election.cc:288] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:43.770577 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a528d9bbfa2488e8ce1e7b99b9e4d49" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:43.770578 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a528d9bbfa2488e8ce1e7b99b9e4d49" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:43.770692 26794 raft_consensus.cc:2454] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:43.770700 27064 raft_consensus.cc:2454] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:43.770958 26861 leader_election.cc:302] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.771138 28789 raft_consensus.cc:2781] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:43.771191 28789 raft_consensus.cc:482] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:43.771234 28789 raft_consensus.cc:3037] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.772078 28789 raft_consensus.cc:504] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.772377 28789 leader_election.cc:288] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:43.772647 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a528d9bbfa2488e8ce1e7b99b9e4d49" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:43.772652 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a528d9bbfa2488e8ce1e7b99b9e4d49" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:43.772749 27064 raft_consensus.cc:3037] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.772753 26794 raft_consensus.cc:3037] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.773451 27064 raft_consensus.cc:2454] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:43.773469 26794 raft_consensus.cc:2454] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:43.773703 26862 leader_election.cc:302] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:43.773923 28789 raft_consensus.cc:2781] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:43.774014 28789 raft_consensus.cc:686] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:43.774181 28789 consensus_queue.cc:237] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.776038 26509 catalog_manager.cc:5484] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:43.781713 28733 raft_consensus.cc:482] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:43.781800 28733 raft_consensus.cc:504] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.782156 28733 leader_election.cc:288] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:43.782423 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b265eec55ee1406b9ff99c159304fa65" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:43.782538 27064 raft_consensus.cc:2454] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:43.782552 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b265eec55ee1406b9ff99c159304fa65" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:43.782660 26930 raft_consensus.cc:2454] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:43.782797 26730 leader_election.cc:302] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.782943 28733 raft_consensus.cc:2781] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:43.782996 28733 raft_consensus.cc:482] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:43.783039 28733 raft_consensus.cc:3037] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.784063 28733 raft_consensus.cc:504] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.784341 28733 leader_election.cc:288] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:43.784606 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b265eec55ee1406b9ff99c159304fa65" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:43.784662 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b265eec55ee1406b9ff99c159304fa65" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:43.784708 27064 raft_consensus.cc:3037] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.784767 26930 raft_consensus.cc:3037] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:43.785396 27064 raft_consensus.cc:2454] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:43.785475 26930 raft_consensus.cc:2454] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:43.785668 26730 leader_election.cc:302] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:43.785809 28733 raft_consensus.cc:2781] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:43.785887 28733 raft_consensus.cc:686] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:43.786015 28733 consensus_queue.cc:237] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:43.787936 26509 catalog_manager.cc:5484] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:43.925241 28811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:43.925455 28811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:43.928416 28811 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_17'
I20240502 01:58:44.012415 27064 raft_consensus.cc:1261] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.012938 28790 consensus_queue.cc:1035] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.013258 26794 raft_consensus.cc:1261] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.014669 28790 consensus_queue.cc:1035] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.020716 26930 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.021072 28653 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.021109 27064 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.021545 28653 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.027594 28774 mvcc.cc:205] Tried to move back new op lower bound from 7023063547996327936 to 7023063546980196352. Current Snapshot: MvccSnapshot[applied={T|T < 7023063547948711936}]
I20240502 01:58:44.036890 26793 raft_consensus.cc:1261] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.037211 26660 raft_consensus.cc:1261] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.038043 28707 consensus_queue.cc:1035] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240502 01:58:44.039160 28707 consensus_queue.cc:1035] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 118.086 ms
time per row: 0.0590431 ms
I20240502 01:58:44.086786 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:44.092272 26641 tablet_service.cc:1467] Processing CreateTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.092584 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d17af0194d5d8c08ad8372b1e9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.092734 26908 tablet_service.cc:1467] Processing CreateTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.092844 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.092991 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d17af0194d5d8c08ad8372b1e9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.093165 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9b0960a2a94c3fb90902c126ccd034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.093664 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 408439eead244f4a8db70600cdc3ce65 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.093870 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408439eead244f4a8db70600cdc3ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.095374 26775 tablet_service.cc:1467] Processing CreateTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.095628 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d17af0194d5d8c08ad8372b1e9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.095820 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 408439eead244f4a8db70600cdc3ce65 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.096029 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408439eead244f4a8db70600cdc3ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.097389 28873 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:44.098434 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.098582 28874 tablet_bootstrap.cc:492] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:44.098667 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9b0960a2a94c3fb90902c126ccd034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.098680 28873 tablet_bootstrap.cc:654] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.099435 28874 tablet_bootstrap.cc:654] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.099623 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.099851 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9b0960a2a94c3fb90902c126ccd034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.100009 28871 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:44.100678 28871 tablet_bootstrap.cc:654] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.101188 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 408439eead244f4a8db70600cdc3ce65 (DEFAULT_TABLE table=rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.101385 28874 tablet_bootstrap.cc:492] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:44.101397 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408439eead244f4a8db70600cdc3ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.101490 28874 ts_tablet_manager.cc:1367] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:44.101821 28872 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:44.101913 28874 raft_consensus.cc:348] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.102018 28874 raft_consensus.cc:374] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.102084 28874 raft_consensus.cc:729] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.102226 28874 consensus_queue.cc:260] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.102506 28874 ts_tablet_manager.cc:1398] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.102850 28872 tablet_bootstrap.cc:654] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.102944 28871 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:44.103030 28871 ts_tablet_manager.cc:1367] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:44.103340 28871 raft_consensus.cc:348] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.103581 28871 raft_consensus.cc:374] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.103636 28871 raft_consensus.cc:729] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.103761 28871 consensus_queue.cc:260] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.103919 28872 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:44.104014 28871 ts_tablet_manager.cc:1398] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:44.104166 28871 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:44.104519 28872 ts_tablet_manager.cc:1367] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:44.104825 28871 tablet_bootstrap.cc:654] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.105058 28872 raft_consensus.cc:348] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.105208 28872 raft_consensus.cc:374] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.105288 28872 raft_consensus.cc:729] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.105460 28872 consensus_queue.cc:260] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.105891 28871 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:44.105967 28871 ts_tablet_manager.cc:1367] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:44.106379 28871 raft_consensus.cc:348] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.106488 28871 raft_consensus.cc:374] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.106534 28871 raft_consensus.cc:729] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.106642 28872 ts_tablet_manager.cc:1398] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:44.106642 28871 consensus_queue.cc:260] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.106894 28872 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:44.106920 28871 ts_tablet_manager.cc:1398] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:44.107067 28871 tablet_bootstrap.cc:492] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:44.107916 28871 tablet_bootstrap.cc:654] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.108108 28872 tablet_bootstrap.cc:654] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.108934 28873 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:44.109019 28873 ts_tablet_manager.cc:1367] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:58:44.109170 28871 tablet_bootstrap.cc:492] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:44.109239 28871 ts_tablet_manager.cc:1367] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:44.109408 28873 raft_consensus.cc:348] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.109503 28873 raft_consensus.cc:374] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.109530 28873 raft_consensus.cc:729] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.109536 28871 raft_consensus.cc:348] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.109601 28871 raft_consensus.cc:374] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.109618 28871 raft_consensus.cc:729] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.109630 28873 consensus_queue.cc:260] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.109722 28871 consensus_queue.cc:260] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.109905 28873 ts_tablet_manager.cc:1398] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:44.110078 28873 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:44.110592 28871 ts_tablet_manager.cc:1398] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:44.110780 28873 tablet_bootstrap.cc:654] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.111929 28873 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:44.112006 28873 ts_tablet_manager.cc:1367] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:44.112404 28873 raft_consensus.cc:348] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.112493 28873 raft_consensus.cc:374] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.112522 28873 raft_consensus.cc:729] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.112623 28873 consensus_queue.cc:260] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.112665 28872 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:44.112735 28872 ts_tablet_manager.cc:1367] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:44.112871 28873 ts_tablet_manager.cc:1398] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:44.113121 28872 raft_consensus.cc:348] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.113204 28872 raft_consensus.cc:374] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.113234 28872 raft_consensus.cc:729] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.113330 28872 consensus_queue.cc:260] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.113584 28872 ts_tablet_manager.cc:1398] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:44.113721 28872 tablet_bootstrap.cc:492] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:44.114539 28872 tablet_bootstrap.cc:654] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.115605 28872 tablet_bootstrap.cc:492] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:44.115687 28872 ts_tablet_manager.cc:1367] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:44.116057 28872 raft_consensus.cc:348] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.116163 28872 raft_consensus.cc:374] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.116194 28872 raft_consensus.cc:729] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.116333 28872 consensus_queue.cc:260] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.116709 28872 ts_tablet_manager.cc:1398] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.147524 28351 raft_consensus.cc:482] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:44.147630 28351 raft_consensus.cc:504] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.147982 28351 leader_election.cc:288] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:44.148277 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408439eead244f4a8db70600cdc3ce65" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:44.148342 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408439eead244f4a8db70600cdc3ce65" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:44.148417 26660 raft_consensus.cc:2454] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:44.148456 26930 raft_consensus.cc:2454] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:44.148677 26997 leader_election.cc:302] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:44.148895 28351 raft_consensus.cc:2781] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:44.148941 28351 raft_consensus.cc:482] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:44.148980 28351 raft_consensus.cc:3037] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.152976 28351 raft_consensus.cc:504] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.153276 28351 leader_election.cc:288] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:44.153635 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408439eead244f4a8db70600cdc3ce65" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:44.153734 26930 raft_consensus.cc:3037] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.153748 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408439eead244f4a8db70600cdc3ce65" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:44.153857 26660 raft_consensus.cc:3037] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.154461 26930 raft_consensus.cc:2454] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:44.154596 26660 raft_consensus.cc:2454] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:44.154738 26998 leader_election.cc:302] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:44.154903 28351 raft_consensus.cc:2781] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:44.154987 28351 raft_consensus.cc:686] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:44.155117 28351 consensus_queue.cc:237] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.156858 26515 catalog_manager.cc:5484] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:44.168633 28653 raft_consensus.cc:482] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:44.168720 28653 raft_consensus.cc:504] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.169034 28653 leader_election.cc:288] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:44.169296 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d17af0194d5d8c08ad8372b1e9c1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:44.169330 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d17af0194d5d8c08ad8372b1e9c1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:44.169392 26930 raft_consensus.cc:2454] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:44.169433 26660 raft_consensus.cc:2454] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:44.169641 26730 leader_election.cc:302] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:44.169792 28653 raft_consensus.cc:2781] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:44.169848 28653 raft_consensus.cc:482] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:44.169891 28653 raft_consensus.cc:3037] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.170754 28653 raft_consensus.cc:504] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.171051 28653 leader_election.cc:288] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:44.171350 26660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d17af0194d5d8c08ad8372b1e9c1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:44.171442 26660 raft_consensus.cc:3037] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.171517 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e2d17af0194d5d8c08ad8372b1e9c1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:44.171622 26930 raft_consensus.cc:3037] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.172159 26660 raft_consensus.cc:2454] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:44.172302 26930 raft_consensus.cc:2454] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:44.172457 26729 leader_election.cc:302] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:44.172699 28653 raft_consensus.cc:2781] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:44.172783 28653 raft_consensus.cc:686] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:44.172905 28653 consensus_queue.cc:237] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.174587 26515 catalog_manager.cc:5484] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:44.207680 28653 raft_consensus.cc:482] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:44.207790 28653 raft_consensus.cc:504] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.208099 28653 leader_election.cc:288] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:44.208500 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9b0960a2a94c3fb90902c126ccd034" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:44.208500 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9b0960a2a94c3fb90902c126ccd034" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:44.208647 26930 raft_consensus.cc:2454] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:44.208647 26659 raft_consensus.cc:2454] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:44.208917 26730 leader_election.cc:302] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:44.209175 28653 raft_consensus.cc:2781] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:44.209231 28653 raft_consensus.cc:482] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:44.209259 28653 raft_consensus.cc:3037] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.210060 28653 raft_consensus.cc:504] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.210393 28653 leader_election.cc:288] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:44.210713 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9b0960a2a94c3fb90902c126ccd034" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:44.210713 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9b0960a2a94c3fb90902c126ccd034" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:44.210834 26659 raft_consensus.cc:3037] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.210834 26930 raft_consensus.cc:3037] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.211793 26659 raft_consensus.cc:2454] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:44.211793 26930 raft_consensus.cc:2454] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:44.212076 26730 leader_election.cc:302] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:44.212255 28653 raft_consensus.cc:2781] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:44.212365 28653 raft_consensus.cc:686] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:44.212503 28653 consensus_queue.cc:237] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:44.214548 26515 catalog_manager.cc:5484] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
W20240502 01:58:44.326531 28887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:44.326748 28887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:44.329900 28887 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_18'
I20240502 01:58:44.401645 26930 raft_consensus.cc:1261] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.402066 28707 consensus_queue.cc:1035] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.404189 26660 raft_consensus.cc:1261] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.405870 26930 raft_consensus.cc:1261] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.407511 28733 consensus_queue.cc:1035] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.408860 28707 consensus_queue.cc:1035] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.409343 26660 raft_consensus.cc:1261] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.410081 28733 consensus_queue.cc:1035] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.414708 26928 raft_consensus.cc:1261] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.415431 26661 raft_consensus.cc:1261] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:44.416424 28653 consensus_queue.cc:1035] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:44.417907 28653 consensus_queue.cc:1035] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 92.0674 ms
time per row: 0.0460337 ms
I20240502 01:58:44.464759 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalance_test_table_19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:44.470300 27044 tablet_service.cc:1467] Processing CreateTablet for tablet dc99f8bfb06c4bcda6111b218125826c (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.470326 26775 tablet_service.cc:1467] Processing CreateTablet for tablet dc99f8bfb06c4bcda6111b218125826c (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.470324 26641 tablet_service.cc:1467] Processing CreateTablet for tablet dc99f8bfb06c4bcda6111b218125826c (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.470611 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc99f8bfb06c4bcda6111b218125826c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.470611 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc99f8bfb06c4bcda6111b218125826c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.470611 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc99f8bfb06c4bcda6111b218125826c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.472544 28874 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:44.472713 28871 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:44.473510 28874 tablet_bootstrap.cc:654] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.473766 28871 tablet_bootstrap.cc:654] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.474007 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 7f5513cf299e44038525ad5f438d74ec (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.474341 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f5513cf299e44038525ad5f438d74ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.474432 28873 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:44.475077 28873 tablet_bootstrap.cc:654] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.475169 28871 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:44.475265 28871 ts_tablet_manager.cc:1367] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:44.475698 28871 raft_consensus.cc:348] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.475769 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 99da1c4053674589b87a7898ed9d57de (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.475808 28871 raft_consensus.cc:374] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.475836 28871 raft_consensus.cc:729] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.475941 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da1c4053674589b87a7898ed9d57de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.475939 28871 consensus_queue.cc:260] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.476224 28871 ts_tablet_manager.cc:1398] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:44.476547 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 7f5513cf299e44038525ad5f438d74ec (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.476785 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f5513cf299e44038525ad5f438d74ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.476848 28872 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:44.477754 28872 tablet_bootstrap.cc:654] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.474007 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 99da1c4053674589b87a7898ed9d57de (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.478344 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da1c4053674589b87a7898ed9d57de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.479077 28872 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:44.479133 28872 ts_tablet_manager.cc:1367] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:44.479485 28872 raft_consensus.cc:348] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.479584 28872 raft_consensus.cc:374] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.479614 28872 raft_consensus.cc:729] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.479697 28872 consensus_queue.cc:260] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.479894 28872 ts_tablet_manager.cc:1398] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:44.480008 28872 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:44.480732 28872 tablet_bootstrap.cc:654] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.482115 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 99da1c4053674589b87a7898ed9d57de (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.482375 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da1c4053674589b87a7898ed9d57de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.484129 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 7f5513cf299e44038525ad5f438d74ec (DEFAULT_TABLE table=rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:44.484395 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f5513cf299e44038525ad5f438d74ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:44.484617 28871 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:44.485278 28874 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:44.485360 28874 ts_tablet_manager.cc:1367] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240502 01:58:44.485409 28873 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:44.485479 28871 tablet_bootstrap.cc:654] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.485483 28873 ts_tablet_manager.cc:1367] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:58:44.485831 28874 raft_consensus.cc:348] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.485931 28874 raft_consensus.cc:374] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.485962 28874 raft_consensus.cc:729] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.485900 28873 raft_consensus.cc:348] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.485996 28873 raft_consensus.cc:374] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.486061 28873 raft_consensus.cc:729] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.486090 28874 consensus_queue.cc:260] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.486176 28873 consensus_queue.cc:260] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.486400 28874 ts_tablet_manager.cc:1398] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.486443 28873 ts_tablet_manager.cc:1398] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.486788 28874 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:44.487191 28871 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:44.487267 28871 ts_tablet_manager.cc:1367] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:44.487637 28874 tablet_bootstrap.cc:654] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.487676 28871 raft_consensus.cc:348] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.487773 28871 raft_consensus.cc:374] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.487811 28871 raft_consensus.cc:729] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.487967 28873 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:44.487926 28871 consensus_queue.cc:260] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.488197 28871 ts_tablet_manager.cc:1398] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:44.488685 28873 tablet_bootstrap.cc:654] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.489953 28874 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:44.490061 28874 ts_tablet_manager.cc:1367] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:58:44.490198 28872 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:44.490309 28872 ts_tablet_manager.cc:1367] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 01:58:44.490448 28874 raft_consensus.cc:348] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.490545 28874 raft_consensus.cc:374] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.490572 28874 raft_consensus.cc:729] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.490679 28874 consensus_queue.cc:260] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.490751 28872 raft_consensus.cc:348] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.490857 28872 raft_consensus.cc:374] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.491039 28874 ts_tablet_manager.cc:1398] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.491209 28874 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:44.491808 28873 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:44.491881 28873 ts_tablet_manager.cc:1367] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:44.492031 28874 tablet_bootstrap.cc:654] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:44.492261 28873 raft_consensus.cc:348] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.492359 28873 raft_consensus.cc:374] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.492388 28873 raft_consensus.cc:729] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.492486 28873 consensus_queue.cc:260] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.492739 28873 ts_tablet_manager.cc:1398] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.493073 28874 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:44.493156 28874 ts_tablet_manager.cc:1367] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:44.493547 28874 raft_consensus.cc:348] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.493640 28874 raft_consensus.cc:374] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:44.493669 28874 raft_consensus.cc:729] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.493667 28872 raft_consensus.cc:729] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:44.493863 28874 consensus_queue.cc:260] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.494153 28874 ts_tablet_manager.cc:1398] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:44.494432 28872 consensus_queue.cc:260] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.494750 28872 ts_tablet_manager.cc:1398] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:44.539887 28920 raft_consensus.cc:482] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:44.540014 28920 raft_consensus.cc:504] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.540434 28920 leader_election.cc:288] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:44.540846 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da1c4053674589b87a7898ed9d57de" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:44.540849 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da1c4053674589b87a7898ed9d57de" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:44.540997 26930 raft_consensus.cc:2454] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:44.540997 27064 raft_consensus.cc:2454] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:44.541260 26595 leader_election.cc:302] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:44.541435 28920 raft_consensus.cc:2781] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:44.541491 28920 raft_consensus.cc:482] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:44.541534 28920 raft_consensus.cc:3037] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.542588 28920 raft_consensus.cc:504] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.542898 28920 leader_election.cc:288] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:44.543264 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da1c4053674589b87a7898ed9d57de" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:44.543385 27064 raft_consensus.cc:3037] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.543486 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99da1c4053674589b87a7898ed9d57de" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:44.543601 26930 raft_consensus.cc:3037] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.544358 27064 raft_consensus.cc:2454] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:44.544473 26930 raft_consensus.cc:2454] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:44.544610 26595 leader_election.cc:302] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:44.544754 28920 raft_consensus.cc:2781] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:44.544840 28920 raft_consensus.cc:686] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:44.544977 28920 consensus_queue.cc:237] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:44.546886 26515 catalog_manager.cc:5484] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:44.567361 28923 raft_consensus.cc:482] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:44.567497 28923 raft_consensus.cc:504] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.567881 28923 leader_election.cc:288] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:44.568217 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc99f8bfb06c4bcda6111b218125826c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:44.568259 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc99f8bfb06c4bcda6111b218125826c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:44.568377 27064 raft_consensus.cc:2454] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:44.568414 26658 raft_consensus.cc:2454] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:44.568724 26729 leader_election.cc:302] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:44.568945 28923 raft_consensus.cc:2781] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:44.569007 28923 raft_consensus.cc:482] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:44.569051 28923 raft_consensus.cc:3037] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.570091 28923 raft_consensus.cc:504] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.570395 28923 leader_election.cc:288] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:44.570720 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc99f8bfb06c4bcda6111b218125826c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:44.570842 26658 raft_consensus.cc:3037] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.571311 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc99f8bfb06c4bcda6111b218125826c" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:44.571429 27064 raft_consensus.cc:3037] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.571789 26658 raft_consensus.cc:2454] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:44.572075 26729 leader_election.cc:302] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:44.572186 27064 raft_consensus.cc:2454] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:44.572250 28653 raft_consensus.cc:2781] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:44.572350 28653 raft_consensus.cc:686] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:44.572487 28653 consensus_queue.cc:237] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.574357 26509 catalog_manager.cc:5484] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:44.840840 28915 raft_consensus.cc:482] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:44.840968 28915 raft_consensus.cc:504] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.841369 28915 leader_election.cc:288] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:44.841722 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f5513cf299e44038525ad5f438d74ec" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:44.841758 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f5513cf299e44038525ad5f438d74ec" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:44.841881 26793 raft_consensus.cc:2454] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:44.841881 26927 raft_consensus.cc:2454] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:44.842221 26998 leader_election.cc:302] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:44.842407 28915 raft_consensus.cc:2781] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:44.842473 28915 raft_consensus.cc:482] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:44.842514 28915 raft_consensus.cc:3037] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.843531 28915 raft_consensus.cc:504] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.843837 28915 leader_election.cc:288] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:44.844069 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f5513cf299e44038525ad5f438d74ec" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:44.844170 26927 raft_consensus.cc:3037] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.844208 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f5513cf299e44038525ad5f438d74ec" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:44.844326 26793 raft_consensus.cc:3037] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:44.844977 26927 raft_consensus.cc:2454] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:44.845002 26793 raft_consensus.cc:2454] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:44.845219 26998 leader_election.cc:302] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:44.845382 28915 raft_consensus.cc:2781] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:44.845465 28915 raft_consensus.cc:686] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:44.845597 28915 consensus_queue.cc:237] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:44.849133 26509 catalog_manager.cc:5484] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:58:45.053982 28941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:45.054261 28941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:45.057569 28941 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 01:58:45.078303 28733 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_19'
I20240502 01:58:45.081622 28733 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:45.087306 28920 consensus_queue.cc:1035] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:45.090318 28808 consensus_queue.cc:1035] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:45.152189 26794 raft_consensus.cc:1261] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:45.152592 28707 consensus_queue.cc:1035] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:45.158867 26927 raft_consensus.cc:1261] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:45.159945 28707 consensus_queue.cc:1035] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 97.4282 ms
time per row: 0.0487141 ms
I20240502 01:58:45.203070 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:34601
--local_ip_for_outbound_sockets=127.19.43.197
--tserver_master_addrs=127.19.43.254:46361
--webserver_port=38069
--webserver_interface=127.19.43.197
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:45.309216 28997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:45.309387 28997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:45.309432 28997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:45.312533 28997 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:45.312592 28997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:45.312667 28997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:58:45.316095 28997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:34601
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=38069
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.28997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:45.316834 28997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:45.317834 28997 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:45.325327 29005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.325330 29002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:45.325466 28997 server_base.cc:1034] running on GCE node
W20240502 01:58:45.325632 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:45.325992 28997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:45.326557 28997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:45.327690 28997 hybrid_clock.cc:648] HybridClock initialized: now 1714615125327675 us; error 28 us; skew 500 ppm
I20240502 01:58:45.329285 28997 webserver.cc:460] Webserver started at http://127.19.43.197:38069/ using document root <none> and password file <none>
I20240502 01:58:45.329782 28997 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:45.329847 28997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:45.332829 28997 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:45.334532 29011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.335506 28997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:45.335598 28997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "f3d18a019013487ab801599cfb80e4b4"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:45.335999 28997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:45.357059 28997 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:45.357561 28997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:45.357690 28997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:45.358158 28997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:45.358934 28997 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:45.358979 28997 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.359025 28997 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:45.359051 28997 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.367429 28997 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:34601
I20240502 01:58:45.367457 29127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:34601 every 8 connection(s)
I20240502 01:58:45.368302 28997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:58:45.376096 29128 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:45.376281 29128 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:45.376664 29128 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:45.377629 26515 ts_manager.cc:194] Re-registered known tserver with Master: f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:45.378240 26515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:36567
I20240502 01:58:45.378460 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 28997
I20240502 01:58:45.378844 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.198:40103
--local_ip_for_outbound_sockets=127.19.43.198
--tserver_master_addrs=127.19.43.254:46361
--webserver_port=36827
--webserver_interface=127.19.43.198
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:58:45.485527 29131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:45.485778 29131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:45.485885 29131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:45.488886 29131 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:45.488945 29131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:45.489006 29131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.198
I20240502 01:58:45.492455 29131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.198:40103
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.19.43.198
--webserver_port=36827
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.29131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.198
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:45.493238 29131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:45.494310 29131 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:45.501493 29140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.501505 29137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.501513 29136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:45.501669 29131 server_base.cc:1034] running on GCE node
I20240502 01:58:45.502069 29131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:45.502633 29131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:45.505988 29131 hybrid_clock.cc:648] HybridClock initialized: now 1714615125505890 us; error 128 us; skew 500 ppm
I20240502 01:58:45.508005 29131 webserver.cc:460] Webserver started at http://127.19.43.198:36827/ using document root <none> and password file <none>
I20240502 01:58:45.508502 29131 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:45.508569 29131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:45.511740 29131 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:58:45.513402 29146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.514318 29131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:45.514415 29131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:45.514783 29131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:45.550284 29131 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:45.551023 29131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:45.551190 29131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:45.551824 29131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:45.552845 29131 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:45.552894 29131 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.552944 29131 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:45.552969 29131 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.561601 29131 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.198:40103
I20240502 01:58:45.561650 29263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.198:40103 every 8 connection(s)
I20240502 01:58:45.562619 29131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 01:58:45.564388 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 29131
I20240502 01:58:45.564842 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.199:44303
--local_ip_for_outbound_sockets=127.19.43.199
--tserver_master_addrs=127.19.43.254:46361
--webserver_port=35609
--webserver_interface=127.19.43.199
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:45.571131 29264 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:45.571323 29264 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:45.571740 29264 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:45.572924 26515 ts_manager.cc:194] Re-registered known tserver with Master: 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:45.573619 26515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.198:47119
W20240502 01:58:45.673960 29267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:45.674170 29267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:45.674221 29267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:45.677243 29267 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:45.677309 29267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:45.677402 29267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.199
I20240502 01:58:45.680809 29267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.199:44303
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.19.43.199
--webserver_port=35609
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.29267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.199
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:45.681466 29267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:45.682412 29267 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:45.689625 29272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.689697 29275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.689731 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:45.689953 29267 server_base.cc:1034] running on GCE node
I20240502 01:58:45.690272 29267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:45.690797 29267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:45.691952 29267 hybrid_clock.cc:648] HybridClock initialized: now 1714615125691935 us; error 32 us; skew 500 ppm
I20240502 01:58:45.693636 29267 webserver.cc:460] Webserver started at http://127.19.43.199:35609/ using document root <none> and password file <none>
I20240502 01:58:45.694182 29267 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:45.694249 29267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:45.697528 29267 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:58:45.699407 29282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.700314 29267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:45.700414 29267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "468dd27456c64d67b22e8a58af93d93d"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:45.700754 29267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:45.735463 29267 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:45.735972 29267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:45.736104 29267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:45.736563 29267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:45.737352 29267 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:45.737399 29267 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.737457 29267 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:45.737486 29267 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.745955 29267 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.199:44303
I20240502 01:58:45.745986 29397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.199:44303 every 8 connection(s)
I20240502 01:58:45.746830 29267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 01:58:45.749996 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 29267
I20240502 01:58:45.750454 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.200:36259
--local_ip_for_outbound_sockets=127.19.43.200
--tserver_master_addrs=127.19.43.254:46361
--webserver_port=35665
--webserver_interface=127.19.43.200
--builtin_ntp_servers=127.19.43.212:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:58:45.755278 29398 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:45.755497 29398 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:45.755951 29398 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:45.756958 26515 ts_manager.cc:194] Re-registered known tserver with Master: 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:45.757453 26515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.199:41523
W20240502 01:58:45.855743 29401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:45.855924 29401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:58:45.855980 29401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:45.858857 29401 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:58:45.858912 29401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:58:45.858989 29401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.200
I20240502 01:58:45.862504 29401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.200:36259
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.19.43.200
--webserver_port=35665
--tserver_master_addrs=127.19.43.254:46361
--never_fsync=true
--heap_profile_path=/tmp/kudu.29401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.200
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:58:45.863204 29401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:58:45.864173 29401 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:58:45.871296 29411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.871302 29408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:58:45.871310 29409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:58:45.871670 29401 server_base.cc:1034] running on GCE node
I20240502 01:58:45.872012 29401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:58:45.872509 29401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:58:45.873715 29401 hybrid_clock.cc:648] HybridClock initialized: now 1714615125873695 us; error 27 us; skew 500 ppm
I20240502 01:58:45.875599 29401 webserver.cc:460] Webserver started at http://127.19.43.200:35665/ using document root <none> and password file <none>
I20240502 01:58:45.876087 29401 fs_manager.cc:359] Metadata directory not provided
I20240502 01:58:45.876157 29401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:58:45.879256 29401 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:58:45.880954 29417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.881840 29401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:45.881932 29401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "7c693d0933c64d9babc07f404df45cd1"
format_stamp: "Formatted at 2024-05-02 01:58:30 on dist-test-slave-mnd3"
I20240502 01:58:45.882315 29401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:58:45.892748 29401 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:58:45.893318 29401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:58:45.893457 29401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:58:45.893918 29401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:58:45.894709 29401 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:58:45.894754 29401 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.894793 29401 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:58:45.894821 29401 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:58:45.903395 29401 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.200:36259
I20240502 01:58:45.903429 29539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.200:36259 every 8 connection(s)
I20240502 01:58:45.904397 29401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 01:58:45.905562 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 29401
I20240502 01:58:45.910189 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:45.915205 29540 heartbeater.cc:347] Connected to a master server at 127.19.43.254:46361
I20240502 01:58:45.915405 29540 heartbeater.cc:464] Registering TS with master...
I20240502 01:58:45.915838 29540 heartbeater.cc:511] Master 127.19.43.254:46361 requested a full tablet report, sending...
I20240502 01:58:45.916891 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 8e548508be244f92bb476bbf83294a80 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]), partition=
I20240502 01:58:45.917156 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e548508be244f92bb476bbf83294a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:45.919368 26506 ts_manager.cc:194] Re-registered known tserver with Master: 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:45.920002 26506 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.200:35207
I20240502 01:58:45.921237 29556 tablet_bootstrap.cc:492] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:45.922124 29556 tablet_bootstrap.cc:654] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:45.925405 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 8e548508be244f92bb476bbf83294a80 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]), partition=
I20240502 01:58:45.925984 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e548508be244f92bb476bbf83294a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:45.930956 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 8e548508be244f92bb476bbf83294a80 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]), partition=
I20240502 01:58:45.931245 29556 tablet_bootstrap.cc:492] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:45.931327 29556 ts_tablet_manager.cc:1367] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 01:58:45.931577 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e548508be244f92bb476bbf83294a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:45.931716 29556 raft_consensus.cc:348] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:45.931813 29556 raft_consensus.cc:374] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:45.931839 29556 raft_consensus.cc:729] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:45.931972 29556 consensus_queue.cc:260] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:45.933219 29556 ts_tablet_manager.cc:1398] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:45.937021 29559 tablet_bootstrap.cc:492] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:45.937515 29558 tablet_bootstrap.cc:492] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:45.939231 29558 tablet_bootstrap.cc:654] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:45.939476 29559 tablet_bootstrap.cc:654] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:45.940012 29558 log.cc:824] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:45.940759 29559 log.cc:824] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:45.941720 29558 tablet_bootstrap.cc:492] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:45.941891 29558 ts_tablet_manager.cc:1367] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 01:58:45.942247 29559 tablet_bootstrap.cc:492] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:45.942409 29559 ts_tablet_manager.cc:1367] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 01:58:45.944928 29559 raft_consensus.cc:348] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:45.945122 29559 raft_consensus.cc:374] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:45.945196 29559 raft_consensus.cc:729] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:45.945477 29558 raft_consensus.cc:348] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:45.945714 29558 raft_consensus.cc:374] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:45.945644 29559 consensus_queue.cc:260] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:45.945801 29558 raft_consensus.cc:729] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:45.946259 29398 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:45.946254 29558 consensus_queue.cc:260] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:45.946481 29559 ts_tablet_manager.cc:1398] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 01:58:45.946964 29128 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:45.946993 29558 ts_tablet_manager.cc:1398] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
W20240502 01:58:45.997718 29399 tablet.cc:2369] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:46.085799 29562 raft_consensus.cc:482] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:46.085968 29562 raft_consensus.cc:504] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.086997 29562 leader_election.cc:288] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:46.091619 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e548508be244f92bb476bbf83294a80" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:46.091861 29080 raft_consensus.cc:2454] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:46.092249 29283 leader_election.cc:302] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:46.092470 29562 raft_consensus.cc:2781] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:46.092545 29562 raft_consensus.cc:482] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:46.092581 29562 raft_consensus.cc:3037] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.093652 29562 raft_consensus.cc:504] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.093998 29562 leader_election.cc:288] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:46.094305 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e548508be244f92bb476bbf83294a80" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:46.094417 29080 raft_consensus.cc:3037] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.094761 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e548508be244f92bb476bbf83294a80" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:46.094867 26658 raft_consensus.cc:3037] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.095463 29080 raft_consensus.cc:2454] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:46.095741 26658 raft_consensus.cc:2454] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:46.096709 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e548508be244f92bb476bbf83294a80" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:46.096822 26661 raft_consensus.cc:2362] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1. Re-sending same reply.
I20240502 01:58:46.097126 29285 leader_election.cc:302] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:46.099334 29562 raft_consensus.cc:2781] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:46.099529 29562 raft_consensus.cc:686] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:46.100023 29562 consensus_queue.cc:237] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.104575 26506 catalog_manager.cc:5484] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
W20240502 01:58:46.119374 29129 tablet.cc:2369] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:46.154464 29196 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.154799 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:46.156600 29058 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.158526 27043 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:58:46.171126 26775 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:58:46.176084 26909 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:46.185748 26641 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:46.199627 29451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.210515 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:46.216244 26658 raft_consensus.cc:1261] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:46.216598 29080 raft_consensus.cc:1261] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:46.216693 29562 consensus_queue.cc:1035] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:46.217293 29562 consensus_queue.cc:1035] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:46.219533 26508 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]). Expected version 1 got 0
I20240502 01:58:46.221112 26507 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]). Expected version 1 got 0
I20240502 01:58:46.222909 29631 mvcc.cc:205] Tried to move back new op lower bound from 7023063556977438720 to 7023063556508749824. Current Snapshot: MvccSnapshot[applied={T|T < 7023063556977438720}]
I20240502 01:58:46.223839 26507 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]). Expected version 1 got 0
I20240502 01:58:46.224294 26508 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb]). Expected version 1 got 0
I20240502 01:58:46.224534 29636 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:46.225282 29636 tablet.cc:1620] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:46.225737 29635 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:46.226368 29635 tablet.cc:1620] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:58:46.228353 29634 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:46.228942 29634 tablet.cc:1620] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:46.231390 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb] alter complete (version 1)
I20240502 01:58:46.335150 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:46.338454 29451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.341635 29196 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.343250 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:46.344199 26909 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:46.347100 29634 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:46.347404 29634 tablet.cc:1620] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:46.347679 29635 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:46.347970 29635 tablet.cc:1620] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:58:46.357589 29636 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:46.357862 29636 tablet.cc:1620] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:46.361835 26775 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:58:46.362440 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=4c7df5b205f148308c18bd697c83e7fb] alter complete (version 2)
I20240502 01:58:46.374459 27043 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:58:46.377444 29058 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.401954 26641 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:46.410389 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 01:58:46.410538 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 01:58:46.410495 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:46.413017 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet e2a7f26d772f43179e473985bc8997d5
I20240502 01:58:46.413350 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 669e26634aaf4fa59df4a3c44772f3a5
I20240502 01:58:46.413605 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 7680bb0330234d0ca29b507a6e329b78
I20240502 01:58:46.413744 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a7f26d772f43179e473985bc8997d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:46.413801 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 669e26634aaf4fa59df4a3c44772f3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:46.414012 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a7f26d772f43179e473985bc8997d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:46.414454 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 669e26634aaf4fa59df4a3c44772f3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:46.414592 29702 tablet_replica.cc:330] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:46.414831 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 7680bb0330234d0ca29b507a6e329b78 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:46.415048 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 7680bb0330234d0ca29b507a6e329b78 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:46.415450 26773 tablet_service.cc:1514] Processing DeleteTablet for tablet 7680bb0330234d0ca29b507a6e329b78 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:46.415422 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 669e26634aaf4fa59df4a3c44772f3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:46.415453 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet e2a7f26d772f43179e473985bc8997d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:46.415789 29704 tablet_replica.cc:330] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:46.416245 29704 raft_consensus.cc:2229] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.416424 29702 raft_consensus.cc:2229] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.416474 29704 raft_consensus.cc:2258] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.416731 29702 raft_consensus.cc:2258] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.417649 29702 ts_tablet_manager.cc:1875] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.417654 29704 ts_tablet_manager.cc:1875] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.419836 26641 tablet_service.cc:1467] Processing CreateTablet for tablet a5bc44bf80f048c3a08e5fb476f7fbbc (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.419934 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 6acba26a44294eb284d03176631ae39b (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.419958 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 6acba26a44294eb284d03176631ae39b (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.420102 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bc44bf80f048c3a08e5fb476f7fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.420239 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acba26a44294eb284d03176631ae39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.420279 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acba26a44294eb284d03176631ae39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.420347 27044 tablet_service.cc:1467] Processing CreateTablet for tablet c46a7ee366464769b8498c4208ce8e9b (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.420559 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46a7ee366464769b8498c4208ce8e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.422300 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 6acba26a44294eb284d03176631ae39b (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.422423 29329 tablet_service.cc:1467] Processing CreateTablet for tablet a5bc44bf80f048c3a08e5fb476f7fbbc (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.422580 29558 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:46.422632 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bc44bf80f048c3a08e5fb476f7fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.423357 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acba26a44294eb284d03176631ae39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.423398 29558 tablet_bootstrap.cc:654] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.423398 26909 tablet_service.cc:1467] Processing CreateTablet for tablet c46a7ee366464769b8498c4208ce8e9b (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.422458 29559 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:46.424248 29556 tablet_bootstrap.cc:492] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:46.424602 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46a7ee366464769b8498c4208ce8e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.424726 29559 tablet_bootstrap.cc:654] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.425062 29556 tablet_bootstrap.cc:654] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.427527 29710 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:46.427738 29702 ts_tablet_manager.cc:1888] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.427855 29702 log.cc:1196] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/e2a7f26d772f43179e473985bc8997d5
I20240502 01:58:46.428213 29702 ts_tablet_manager.cc:1909] T e2a7f26d772f43179e473985bc8997d5 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:46.428385 29710 tablet_bootstrap.cc:654] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.429536 29702 tablet_replica.cc:330] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:46.429735 29705 tablet_replica.cc:330] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:46.430068 29705 raft_consensus.cc:2229] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.430323 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet e2a7f26d772f43179e473985bc8997d5 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.430779 29558 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:46.430861 29558 ts_tablet_manager.cc:1367] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 01:58:46.430830 29196 tablet_service.cc:1467] Processing CreateTablet for tablet c46a7ee366464769b8498c4208ce8e9b (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.430830 29195 tablet_service.cc:1467] Processing CreateTablet for tablet a5bc44bf80f048c3a08e5fb476f7fbbc (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:46.431365 29558 raft_consensus.cc:348] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.431464 29558 raft_consensus.cc:374] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.431492 29558 raft_consensus.cc:729] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.431612 29558 consensus_queue.cc:260] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.431810 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bc44bf80f048c3a08e5fb476f7fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.431869 29558 ts_tablet_manager.cc:1398] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:46.432371 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46a7ee366464769b8498c4208ce8e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.432497 29705 raft_consensus.cc:2258] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.432621 29704 ts_tablet_manager.cc:1888] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.432687 29704 log.cc:1196] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/e2a7f26d772f43179e473985bc8997d5
I20240502 01:58:46.433014 29704 ts_tablet_manager.cc:1909] T e2a7f26d772f43179e473985bc8997d5 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:46.433471 29705 ts_tablet_manager.cc:1875] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.433892 29704 tablet_replica.cc:330] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:46.433948 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet e2a7f26d772f43179e473985bc8997d5 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.434175 29706 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:46.434325 29702 raft_consensus.cc:2229] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.434489 29702 raft_consensus.cc:2258] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.435070 29702 ts_tablet_manager.cc:1875] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.435392 29706 tablet_bootstrap.cc:654] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.435850 29559 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:46.435940 29559 ts_tablet_manager.cc:1367] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:46.436426 29559 raft_consensus.cc:348] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.436522 29559 raft_consensus.cc:374] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.436565 29559 raft_consensus.cc:729] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.436683 29559 consensus_queue.cc:260] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.436929 29559 ts_tablet_manager.cc:1398] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:46.437077 29559 tablet_bootstrap.cc:492] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:46.437119 29703 tablet_replica.cc:330] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:46.437515 29703 raft_consensus.cc:2229] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.437834 29703 raft_consensus.cc:2258] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.437908 29559 tablet_bootstrap.cc:654] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.438421 29702 ts_tablet_manager.cc:1888] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.438482 29702 log.cc:1196] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/669e26634aaf4fa59df4a3c44772f3a5
I20240502 01:58:46.438663 29703 ts_tablet_manager.cc:1875] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.438711 29702 ts_tablet_manager.cc:1909] T 669e26634aaf4fa59df4a3c44772f3a5 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:46.440101 29556 tablet_bootstrap.cc:492] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:46.440186 29556 ts_tablet_manager.cc:1367] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.002s
I20240502 01:58:46.440223 29704 raft_consensus.cc:2229] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.440586 29556 raft_consensus.cc:348] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.440685 29556 raft_consensus.cc:374] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.440711 29556 raft_consensus.cc:729] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.440735 29704 raft_consensus.cc:2258] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.440852 29556 consensus_queue.cc:260] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.441123 29556 ts_tablet_manager.cc:1398] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:46.441287 29704 ts_tablet_manager.cc:1875] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.443374 29714 tablet_bootstrap.cc:492] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:46.444003 29704 ts_tablet_manager.cc:1888] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.444077 29704 log.cc:1196] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/7680bb0330234d0ca29b507a6e329b78
I20240502 01:58:46.444319 29704 ts_tablet_manager.cc:1909] T 7680bb0330234d0ca29b507a6e329b78 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:46.445446 29714 tablet_bootstrap.cc:654] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.445508 29559 tablet_bootstrap.cc:492] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:46.445592 29559 ts_tablet_manager.cc:1367] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 01:58:46.445996 29559 raft_consensus.cc:348] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.446141 29559 raft_consensus.cc:374] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.446178 29559 raft_consensus.cc:729] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.446177 29703 ts_tablet_manager.cc:1888] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.446256 29703 log.cc:1196] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/669e26634aaf4fa59df4a3c44772f3a5
I20240502 01:58:46.446300 29559 consensus_queue.cc:260] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.446553 29559 ts_tablet_manager.cc:1398] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:46.446537 29714 log.cc:824] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:46.446667 29703 ts_tablet_manager.cc:1909] T 669e26634aaf4fa59df4a3c44772f3a5 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:46.447567 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 669e26634aaf4fa59df4a3c44772f3a5 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.447845 29703 tablet_replica.cc:330] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:46.448014 29703 raft_consensus.cc:2229] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.448032 29706 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:46.448112 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 7680bb0330234d0ca29b507a6e329b78 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.448101 29706 ts_tablet_manager.cc:1367] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240502 01:58:46.448278 29703 raft_consensus.cc:2258] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.448508 29706 raft_consensus.cc:348] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.448660 29706 raft_consensus.cc:374] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.448731 29706 raft_consensus.cc:729] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.448777 29703 ts_tablet_manager.cc:1875] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.448920 29706 consensus_queue.cc:260] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.449352 29706 ts_tablet_manager.cc:1398] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:46.449533 29706 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:46.450361 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 669e26634aaf4fa59df4a3c44772f3a5 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.450598 29705 ts_tablet_manager.cc:1888] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.450661 29703 ts_tablet_manager.cc:1888] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.450675 29705 log.cc:1196] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/7680bb0330234d0ca29b507a6e329b78
I20240502 01:58:46.450712 29703 log.cc:1196] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/7680bb0330234d0ca29b507a6e329b78
I20240502 01:58:46.450954 29703 ts_tablet_manager.cc:1909] T 7680bb0330234d0ca29b507a6e329b78 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:46.450956 29705 ts_tablet_manager.cc:1909] T 7680bb0330234d0ca29b507a6e329b78 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:46.451539 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 7680bb0330234d0ca29b507a6e329b78 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.451871 29705 tablet_replica.cc:330] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:46.451925 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 7680bb0330234d0ca29b507a6e329b78 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.451997 29705 raft_consensus.cc:2229] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.452107 29705 raft_consensus.cc:2258] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.452337 29706 tablet_bootstrap.cc:654] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.452581 29705 ts_tablet_manager.cc:1875] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.453274 29710 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:46.453382 29710 ts_tablet_manager.cc:1367] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20240502 01:58:46.453787 29710 raft_consensus.cc:348] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.453882 29710 raft_consensus.cc:374] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.453912 29710 raft_consensus.cc:729] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.454051 29710 consensus_queue.cc:260] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.454123 29705 ts_tablet_manager.cc:1888] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.454286 29706 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:46.454329 29710 ts_tablet_manager.cc:1398] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:46.454422 29706 ts_tablet_manager.cc:1367] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:46.454972 29706 raft_consensus.cc:348] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.455075 29706 raft_consensus.cc:374] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.455137 29706 raft_consensus.cc:729] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.455248 29706 consensus_queue.cc:260] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.455484 29706 ts_tablet_manager.cc:1398] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:46.460220 29705 log.cc:1196] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/669e26634aaf4fa59df4a3c44772f3a5
I20240502 01:58:46.460462 29705 ts_tablet_manager.cc:1909] T 669e26634aaf4fa59df4a3c44772f3a5 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:46.460875 29705 tablet_replica.cc:330] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:46.460951 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 669e26634aaf4fa59df4a3c44772f3a5 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.461007 29705 raft_consensus.cc:2229] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.461115 29705 raft_consensus.cc:2258] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.461488 29705 ts_tablet_manager.cc:1875] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.462929 29705 ts_tablet_manager.cc:1888] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.462997 29705 log.cc:1196] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/e2a7f26d772f43179e473985bc8997d5
I20240502 01:58:46.463209 29705 ts_tablet_manager.cc:1909] T e2a7f26d772f43179e473985bc8997d5 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:46.463367 29714 tablet_bootstrap.cc:492] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:46.463531 29714 ts_tablet_manager.cc:1367] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20240502 01:58:46.463789 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet e2a7f26d772f43179e473985bc8997d5 (table rebalance_test_table_0 [id=57c662fbb7414863ae47f44c2b52d38a]) successfully deleted
I20240502 01:58:46.466594 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:46.466660 29714 raft_consensus.cc:348] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.466858 29714 raft_consensus.cc:374] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.466909 29714 raft_consensus.cc:729] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.467339 29714 consensus_queue.cc:260] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.468165 29714 ts_tablet_manager.cc:1398] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 01:58:46.468225 29264 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:46.468382 29714 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:46.469806 29714 tablet_bootstrap.cc:654] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.471001 29714 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:46.471069 29714 ts_tablet_manager.cc:1367] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:46.471530 29714 raft_consensus.cc:348] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.471614 29714 raft_consensus.cc:374] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.471647 29714 raft_consensus.cc:729] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.471748 29714 consensus_queue.cc:260] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.471984 29714 ts_tablet_manager.cc:1398] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:46.475507 29634 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:46.476482 29636 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:46.477181 29635 tablet.cc:1722] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:46.479424 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=4c7df5b205f148308c18bd697c83e7fb] alter complete (version 3)
I20240502 01:58:46.500982 29563 raft_consensus.cc:482] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:46.501070 29563 raft_consensus.cc:504] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.501897 29563 leader_election.cc:288] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:46.505930 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6acba26a44294eb284d03176631ae39b" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:46.506107 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6acba26a44294eb284d03176631ae39b" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:46.506424 27063 raft_consensus.cc:2454] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:46.506343 29350 raft_consensus.cc:2454] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:46.506815 29016 leader_election.cc:302] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:46.507009 29563 raft_consensus.cc:2781] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:46.507066 29563 raft_consensus.cc:482] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:46.507089 29563 raft_consensus.cc:3037] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.507786 29563 raft_consensus.cc:504] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.508088 29563 leader_election.cc:288] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:46.508365 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6acba26a44294eb284d03176631ae39b" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:46.508397 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6acba26a44294eb284d03176631ae39b" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:46.508459 27063 raft_consensus.cc:3037] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.508479 29350 raft_consensus.cc:3037] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.509176 27063 raft_consensus.cc:2454] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:46.509224 29350 raft_consensus.cc:2454] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:46.509472 29016 leader_election.cc:302] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:46.509631 29563 raft_consensus.cc:2781] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:46.509812 29563 raft_consensus.cc:686] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:46.510098 29563 consensus_queue.cc:237] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.511642 26514 catalog_manager.cc:5484] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
W20240502 01:58:46.563894 29265 tablet.cc:2369] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:46.579581 29726 raft_consensus.cc:482] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:46.579728 29726 raft_consensus.cc:504] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.580783 29726 leader_election.cc:288] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:46.582361 26512 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:46.582459 26512 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:46.585419 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 8e548508be244f92bb476bbf83294a80
I20240502 01:58:46.586048 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e548508be244f92bb476bbf83294a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:46.585934 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e548508be244f92bb476bbf83294a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:46.586275 29703 tablet_replica.cc:330] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:46.586410 29703 raft_consensus.cc:2229] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.586519 29703 raft_consensus.cc:2258] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.586956 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bc44bf80f048c3a08e5fb476f7fbbc" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:46.587080 26658 raft_consensus.cc:2454] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:46.587193 29767 tablet_replica.cc:330] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:46.587327 29329 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e548508be244f92bb476bbf83294a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:46.587332 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bc44bf80f048c3a08e5fb476f7fbbc" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:46.587390 29149 leader_election.cc:302] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:46.587493 29350 raft_consensus.cc:2454] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:46.587697 29726 raft_consensus.cc:2781] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:46.587743 29768 tablet_replica.cc:330] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:46.587767 29726 raft_consensus.cc:482] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:46.587816 29726 raft_consensus.cc:3037] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.587997 29703 ts_tablet_manager.cc:1875] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.588002 29768 raft_consensus.cc:2229] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.588356 29768 raft_consensus.cc:2258] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.588809 29726 raft_consensus.cc:504] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.589131 29726 leader_election.cc:288] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:46.589263 29768 ts_tablet_manager.cc:1875] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.589519 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bc44bf80f048c3a08e5fb476f7fbbc" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:46.589622 29350 raft_consensus.cc:3037] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.589694 29767 raft_consensus.cc:2229] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.589871 29767 raft_consensus.cc:2258] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.589905 29703 ts_tablet_manager.cc:1888] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:46.589963 29703 log.cc:1196] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/8e548508be244f92bb476bbf83294a80
I20240502 01:58:46.590214 29703 ts_tablet_manager.cc:1909] T 8e548508be244f92bb476bbf83294a80 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:46.590538 29350 raft_consensus.cc:2454] T a5bc44bf80f048c3a08e5fb476f7fbbc P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:46.590873 29147 leader_election.cc:302] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:46.591041 29726 raft_consensus.cc:2781] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:46.591096 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table_new_name [id=4c7df5b205f148308c18bd697c83e7fb]) successfully deleted
I20240502 01:58:46.591240 29726 raft_consensus.cc:686] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:46.591270 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bc44bf80f048c3a08e5fb476f7fbbc" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:46.591379 26658 raft_consensus.cc:3037] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.591538 29726 consensus_queue.cc:237] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.591989 26658 raft_consensus.cc:2454] T a5bc44bf80f048c3a08e5fb476f7fbbc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:46.592028 29768 ts_tablet_manager.cc:1888] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:46.592154 29768 log.cc:1196] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/8e548508be244f92bb476bbf83294a80
I20240502 01:58:46.592456 29768 ts_tablet_manager.cc:1909] T 8e548508be244f92bb476bbf83294a80 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:46.592707 29767 ts_tablet_manager.cc:1875] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.593987 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table_new_name [id=4c7df5b205f148308c18bd697c83e7fb]) successfully deleted
I20240502 01:58:46.594874 29767 ts_tablet_manager.cc:1888] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:46.595007 29767 log.cc:1196] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/8e548508be244f92bb476bbf83294a80
I20240502 01:58:46.595337 29767 ts_tablet_manager.cc:1909] T 8e548508be244f92bb476bbf83294a80 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:46.596242 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 8e548508be244f92bb476bbf83294a80 (table rebalancer_dynamic_table_new_name [id=4c7df5b205f148308c18bd697c83e7fb]) successfully deleted
I20240502 01:58:46.596475 26512 catalog_manager.cc:5484] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.624490 28915 raft_consensus.cc:482] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:46.624573 28915 raft_consensus.cc:504] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.624951 28915 leader_election.cc:288] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:46.625267 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46a7ee366464769b8498c4208ce8e9b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:46.625394 26930 raft_consensus.cc:2454] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:46.625874 26998 leader_election.cc:302] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:46.626016 28915 raft_consensus.cc:2781] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:46.626096 28915 raft_consensus.cc:482] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:46.626122 28915 raft_consensus.cc:3037] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.626874 28915 raft_consensus.cc:504] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.627188 28915 leader_election.cc:288] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:46.627502 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46a7ee366464769b8498c4208ce8e9b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:46.627622 26930 raft_consensus.cc:3037] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.628434 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46a7ee366464769b8498c4208ce8e9b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:46.628504 26930 raft_consensus.cc:2454] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:46.628440 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46a7ee366464769b8498c4208ce8e9b" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:46.628639 29213 raft_consensus.cc:2454] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:46.628815 26998 leader_election.cc:302] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:46.628973 28915 raft_consensus.cc:2781] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:46.629053 28915 raft_consensus.cc:686] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:46.629173 28915 consensus_queue.cc:237] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:46.630898 26513 catalog_manager.cc:5484] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:46.648728 29196 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:46.649223 26641 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:46.650768 29329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.652527 29451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:46.657198 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:58:46.666241 27043 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 01:58:46.667665 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:46.675848 29058 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:46.687755 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:46.693706 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 2c40d8b6702b4265a71986b0102b6268 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774]), partition=
I20240502 01:58:46.693976 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c40d8b6702b4265a71986b0102b6268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.694352 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 2c40d8b6702b4265a71986b0102b6268 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774]), partition=
I20240502 01:58:46.694554 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c40d8b6702b4265a71986b0102b6268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.696461 29710 tablet_bootstrap.cc:492] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:46.696635 29558 tablet_bootstrap.cc:492] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:46.697321 29558 tablet_bootstrap.cc:654] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.697321 29710 tablet_bootstrap.cc:654] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.704617 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 2c40d8b6702b4265a71986b0102b6268 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774]), partition=
I20240502 01:58:46.705363 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c40d8b6702b4265a71986b0102b6268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:46.707932 29558 tablet_bootstrap.cc:492] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:46.708022 29558 ts_tablet_manager.cc:1367] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:58:46.708459 29558 raft_consensus.cc:348] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.708586 29558 raft_consensus.cc:374] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.708619 29558 raft_consensus.cc:729] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.708724 29558 consensus_queue.cc:260] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.708992 29558 ts_tablet_manager.cc:1398] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:46.709231 29710 tablet_bootstrap.cc:492] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:46.709323 29710 ts_tablet_manager.cc:1367] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:58:46.709738 29710 raft_consensus.cc:348] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.709856 29710 raft_consensus.cc:374] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.709887 29710 raft_consensus.cc:729] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.709995 29710 consensus_queue.cc:260] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.710281 29710 ts_tablet_manager.cc:1398] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:46.715606 29786 tablet_bootstrap.cc:492] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:46.717270 29786 tablet_bootstrap.cc:654] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:46.718050 29786 log.cc:824] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Log is configured to *not* fsync() on all Append() calls
I20240502 01:58:46.719465 29786 tablet_bootstrap.cc:492] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:46.719616 29786 ts_tablet_manager.cc:1367] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:58:46.722432 29786 raft_consensus.cc:348] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.722641 29786 raft_consensus.cc:374] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:46.722739 29786 raft_consensus.cc:729] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:46.723204 29786 consensus_queue.cc:260] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.723779 29540 heartbeater.cc:503] Master 127.19.43.254:46361 was elected leader, sending a full tablet report...
I20240502 01:58:46.724011 29786 ts_tablet_manager.cc:1398] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 01:58:46.756100 27064 consensus_queue.cc:237] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.757496 26658 raft_consensus.cc:1261] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.757843 26793 raft_consensus.cc:1261] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.758244 29625 consensus_queue.cc:1035] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.758584 28915 consensus_queue.cc:1035] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:46.761915 26998 consensus_peers.cc:485] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5439ff11ca04925bf46e736a5a53487. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.763908 29625 raft_consensus.cc:2932] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.764283 26793 raft_consensus.cc:2932] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.766078 26658 raft_consensus.cc:2932] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.767664 26512 catalog_manager.cc:5484] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.784893 26793 consensus_queue.cc:237] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:46.786345 27064 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.786687 26930 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.786749 29826 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.787082 29564 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.789702 29826 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:46.790241 26927 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
W20240502 01:58:46.792121 26729 consensus_peers.cc:485] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b265eec55ee1406b9ff99c159304fa65. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.792357 26513 catalog_manager.cc:5484] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:46.794348 27064 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:46.800817 28924 raft_consensus.cc:482] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:46.800937 28924 raft_consensus.cc:504] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.801501 28924 leader_election.cc:288] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:46.811821 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c40d8b6702b4265a71986b0102b6268" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:46.811960 29080 raft_consensus.cc:2454] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:46.812629 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c40d8b6702b4265a71986b0102b6268" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:46.812933 29489 raft_consensus.cc:2454] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:46.813382 26862 leader_election.cc:302] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:46.813541 28924 raft_consensus.cc:2781] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:46.813585 28924 raft_consensus.cc:482] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:46.813616 28924 raft_consensus.cc:3037] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.814512 28924 raft_consensus.cc:504] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.814821 28924 leader_election.cc:288] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:46.815184 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c40d8b6702b4265a71986b0102b6268" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:46.815227 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c40d8b6702b4265a71986b0102b6268" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:46.815294 29080 raft_consensus.cc:3037] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.815336 29489 raft_consensus.cc:3037] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:46.816151 29080 raft_consensus.cc:2454] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:46.816406 29489 raft_consensus.cc:2454] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:46.816586 26859 leader_election.cc:302] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:46.817050 26793 consensus_queue.cc:237] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.818403 26927 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.818466 27064 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.818990 28733 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.819305 28733 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.819541 28924 raft_consensus.cc:2781] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:46.819620 28924 raft_consensus.cc:686] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:46.819727 28924 consensus_queue.cc:237] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
W20240502 01:58:46.822405 26727 consensus_peers.cc:485] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 565aab51035c4f328e49eca53eddf621. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.822530 26507 catalog_manager.cc:5484] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.823218 28733 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.825170 26930 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.825929 27064 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.828320 26507 catalog_manager.cc:5484] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.845598 27064 consensus_queue.cc:237] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.846805 26930 raft_consensus.cc:1261] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.846818 26793 raft_consensus.cc:1261] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.847170 28915 consensus_queue.cc:1035] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.847342 29806 consensus_queue.cc:1035] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.849864 28915 raft_consensus.cc:2932] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
W20240502 01:58:46.850286 26995 consensus_peers.cc:485] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cceb8a91997e4db9a4842df26ad3b02e. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.850335 26793 raft_consensus.cc:2932] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.851629 26930 raft_consensus.cc:2932] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.854174 26510 catalog_manager.cc:5484] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.869716 26793 consensus_queue.cc:237] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.870997 26658 raft_consensus.cc:1261] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.871239 26930 raft_consensus.cc:1261] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.871448 29851 consensus_queue.cc:1035] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.871613 29564 consensus_queue.cc:1035] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.873497 29564 raft_consensus.cc:2932] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.873839 26658 raft_consensus.cc:2932] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
W20240502 01:58:46.874658 26730 consensus_peers.cc:485] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6e2ebe00a52240cb9ac5fc8fea9d7151. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.875267 26930 raft_consensus.cc:2932] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.876161 26512 catalog_manager.cc:5484] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.893436 26793 consensus_queue.cc:237] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:46.894640 27064 raft_consensus.cc:1261] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.894658 26658 raft_consensus.cc:1261] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:46.894850 26729 consensus_peers.cc:485] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c002ce2d74547399685b69a0a4cbada. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.895052 29826 consensus_queue.cc:1035] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.895231 29851 consensus_queue.cc:1035] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.896788 29826 raft_consensus.cc:2932] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:46.897192 27064 raft_consensus.cc:2932] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:46.898620 26658 raft_consensus.cc:2932] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:46.900092 26510 catalog_manager.cc:5484] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
W20240502 01:58:46.907569 29541 tablet.cc:2369] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:58:46.915047 26507 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 01:58:46.915175 26507 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 01:58:46.917310 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet a42923ea9b5f438a8760fcff8b93e288
I20240502 01:58:46.917562 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 19295fe971e7409384b39937ea5d80e3
I20240502 01:58:46.917524 27064 consensus_queue.cc:237] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:46.917804 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet c618121cd88d4a53a6badd33b99c439c
I20240502 01:58:46.917842 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet a42923ea9b5f438a8760fcff8b93e288 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:46.917991 29704 tablet_replica.cc:330] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:46.918138 29704 raft_consensus.cc:2229] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.918237 29704 raft_consensus.cc:2258] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.918478 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 19295fe971e7409384b39937ea5d80e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:46.918702 29704 ts_tablet_manager.cc:1875] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.918833 29702 tablet_replica.cc:330] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:46.918951 29702 raft_consensus.cc:2229] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.919080 29702 raft_consensus.cc:2258] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.919091 26930 raft_consensus.cc:1261] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.919431 29625 consensus_queue.cc:1035] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.919515 29702 ts_tablet_manager.cc:1875] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.919757 26658 raft_consensus.cc:1261] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.919780 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet c618121cd88d4a53a6badd33b99c439c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:46.920743 29704 ts_tablet_manager.cc:1888] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.920814 29704 log.cc:1196] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/a42923ea9b5f438a8760fcff8b93e288
I20240502 01:58:46.920944 29625 consensus_queue.cc:1035] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.921041 29704 ts_tablet_manager.cc:1909] T a42923ea9b5f438a8760fcff8b93e288 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:46.921244 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet a42923ea9b5f438a8760fcff8b93e288 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:46.921567 29703 tablet_replica.cc:330] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:46.921691 29702 ts_tablet_manager.cc:1888] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.921705 29703 raft_consensus.cc:2229] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.921746 29702 log.cc:1196] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/19295fe971e7409384b39937ea5d80e3
I20240502 01:58:46.921833 29703 raft_consensus.cc:2258] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.921969 29702 ts_tablet_manager.cc:1909] T 19295fe971e7409384b39937ea5d80e3 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:46.922318 29703 ts_tablet_manager.cc:1875] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.922905 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 19295fe971e7409384b39937ea5d80e3 (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.923959 29703 ts_tablet_manager.cc:1888] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.924005 29703 log.cc:1196] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/a42923ea9b5f438a8760fcff8b93e288
I20240502 01:58:46.924160 29703 ts_tablet_manager.cc:1909] T a42923ea9b5f438a8760fcff8b93e288 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:46.924563 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 19295fe971e7409384b39937ea5d80e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:46.925087 26773 tablet_service.cc:1514] Processing DeleteTablet for tablet a42923ea9b5f438a8760fcff8b93e288 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:46.925204 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet c618121cd88d4a53a6badd33b99c439c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:46.925247 29705 tablet_replica.cc:330] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:46.925311 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet c618121cd88d4a53a6badd33b99c439c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:46.925213 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 19295fe971e7409384b39937ea5d80e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:46 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:46.925469 29703 tablet_replica.cc:330] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:46.925392 29705 raft_consensus.cc:2229] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.925586 29703 raft_consensus.cc:2229] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.925688 29703 raft_consensus.cc:2258] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.925675 29705 raft_consensus.cc:2258] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.925757 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet a42923ea9b5f438a8760fcff8b93e288 (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.926164 29705 ts_tablet_manager.cc:1875] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.926654 29704 tablet_replica.cc:330] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:46.926744 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet a42923ea9b5f438a8760fcff8b93e288 (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.926874 29704 raft_consensus.cc:2229] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.927069 29704 raft_consensus.cc:2258] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.927261 29703 ts_tablet_manager.cc:1875] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240502 01:58:46.928086 26995 consensus_peers.cc:485] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 936e77ced55c41d6bfe955d9a4271b80. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.928151 29704 ts_tablet_manager.cc:1875] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.928195 29705 ts_tablet_manager.cc:1888] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.928256 29705 log.cc:1196] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/a42923ea9b5f438a8760fcff8b93e288
I20240502 01:58:46.928260 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:46.928498 29705 ts_tablet_manager.cc:1909] T a42923ea9b5f438a8760fcff8b93e288 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:46.928997 29705 tablet_replica.cc:330] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:46.929068 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet a42923ea9b5f438a8760fcff8b93e288 (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.929116 29705 raft_consensus.cc:2229] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.929217 29705 raft_consensus.cc:2258] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.929461 29703 ts_tablet_manager.cc:1888] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.929574 29703 log.cc:1196] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/c618121cd88d4a53a6badd33b99c439c
I20240502 01:58:46.929654 29705 ts_tablet_manager.cc:1875] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.929908 29703 ts_tablet_manager.cc:1909] T c618121cd88d4a53a6badd33b99c439c P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:46.930507 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet c618121cd88d4a53a6badd33b99c439c (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.930997 29704 ts_tablet_manager.cc:1888] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.931066 29704 log.cc:1196] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/c618121cd88d4a53a6badd33b99c439c
I20240502 01:58:46.931339 29704 ts_tablet_manager.cc:1909] T c618121cd88d4a53a6badd33b99c439c P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:46.931876 29704 tablet_replica.cc:330] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:46.931975 29705 ts_tablet_manager.cc:1888] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.931977 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet c618121cd88d4a53a6badd33b99c439c (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.932039 29705 log.cc:1196] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/c618121cd88d4a53a6badd33b99c439c
I20240502 01:58:46.932185 29704 raft_consensus.cc:2229] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:46.932260 29705 ts_tablet_manager.cc:1909] T c618121cd88d4a53a6badd33b99c439c P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:46.932390 29704 raft_consensus.cc:2258] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.932724 29705 tablet_replica.cc:330] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:46.932822 29705 raft_consensus.cc:2229] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:46.932818 29080 raft_consensus.cc:1261] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:46.932829 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet c618121cd88d4a53a6badd33b99c439c (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.932936 29705 raft_consensus.cc:2258] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:46.933065 29489 raft_consensus.cc:1261] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:46.933156 29704 ts_tablet_manager.cc:1875] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.933226 29565 consensus_queue.cc:1035] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:46.933328 29705 ts_tablet_manager.cc:1875] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:46.934360 29565 consensus_queue.cc:1035] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:46.935051 29705 ts_tablet_manager.cc:1888] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.935122 29705 log.cc:1196] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/19295fe971e7409384b39937ea5d80e3
I20240502 01:58:46.935364 29705 ts_tablet_manager.cc:1909] T 19295fe971e7409384b39937ea5d80e3 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:46.935839 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 19295fe971e7409384b39937ea5d80e3 (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.936935 26507 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 2c40d8b6702b4265a71986b0102b6268 (table rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774]). Expected version 1 got 0
I20240502 01:58:46.937608 29704 ts_tablet_manager.cc:1888] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:46.937711 29704 log.cc:1196] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/19295fe971e7409384b39937ea5d80e3
I20240502 01:58:46.937995 29704 ts_tablet_manager.cc:1909] T 19295fe971e7409384b39937ea5d80e3 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:46.938271 26512 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 2c40d8b6702b4265a71986b0102b6268 (table rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774]). Expected version 1 got 0
I20240502 01:58:46.938745 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 19295fe971e7409384b39937ea5d80e3 (table rebalance_test_table_2 [id=8bd0d307cf9b4c90b9f1bc7cc8fdeb01]) successfully deleted
I20240502 01:58:46.940291 29806 raft_consensus.cc:2932] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.940605 26930 raft_consensus.cc:2932] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.942593 29636 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:46.942890 29636 tablet.cc:1620] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:46.943396 26510 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 2c40d8b6702b4265a71986b0102b6268 (table rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774]). Expected version 1 got 0
I20240502 01:58:46.943539 26507 catalog_manager.cc:5484] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.945036 26658 raft_consensus.cc:2932] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:46.947868 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774] alter complete (version 1)
I20240502 01:58:46.944835 29929 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:46.949877 29929 tablet.cc:1620] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:46.950203 29930 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:46.950956 29930 tablet.cc:1620] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:46.964182 27064 consensus_queue.cc:237] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:46.965420 26793 raft_consensus.cc:1261] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.965513 26930 raft_consensus.cc:1261] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.965812 28915 consensus_queue.cc:1035] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:46.965970 26995 consensus_peers.cc:485] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f5513cf299e44038525ad5f438d74ec. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.965983 29625 consensus_queue.cc:1035] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.967770 29625 raft_consensus.cc:2932] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.968191 26793 raft_consensus.cc:2932] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.969345 26930 raft_consensus.cc:2932] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.970357 26507 catalog_manager.cc:5484] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:46.990204 27064 consensus_queue.cc:237] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:46.991349 26930 raft_consensus.cc:1261] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:46.991452 26658 raft_consensus.cc:1261] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:46.991598 26998 consensus_peers.cc:485] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e812884ef6474b5384d026637dc62b61. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:46.991825 29809 consensus_queue.cc:1035] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.991986 28915 consensus_queue.cc:1035] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:46.993541 29809 raft_consensus.cc:2932] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.993952 26930 raft_consensus.cc:2932] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.995513 26658 raft_consensus.cc:2932] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:46.996044 26510 catalog_manager.cc:5484] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.012434 26793 consensus_queue.cc:237] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.013567 27064 raft_consensus.cc:1261] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.013626 26658 raft_consensus.cc:1261] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:47.013942 26729 consensus_peers.cc:485] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdd32c8294b44540ac5b832cc48d2519. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.013975 29826 consensus_queue.cc:1035] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.014161 29851 consensus_queue.cc:1035] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.016001 29216 raft_consensus.cc:3037] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.016897 29851 raft_consensus.cc:2932] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.017377 26658 raft_consensus.cc:2932] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.018273 29809 consensus_queue.cc:1035] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.019974 26513 catalog_manager.cc:5484] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.021013 27064 raft_consensus.cc:2932] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.023053 29563 consensus_queue.cc:1035] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.025648 29625 consensus_queue.cc:1035] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.028084 29997 consensus_queue.cc:1035] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.043133 26793 consensus_queue.cc:237] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.044672 26927 raft_consensus.cc:1261] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.044971 26658 raft_consensus.cc:1261] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.045176 29564 consensus_queue.cc:1035] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.045284 29726 consensus_queue.cc:1035] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.045634 29826 consensus_queue.cc:1035] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.048396 29826 raft_consensus.cc:2932] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
W20240502 01:58:47.049326 26727 consensus_peers.cc:485] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce3d632b976e491e8ac726952c69cbc3. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.049852 26927 raft_consensus.cc:2932] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.050112 26658 raft_consensus.cc:2932] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.051090 26512 catalog_manager.cc:5484] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.052433 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:47.054056 29726 consensus_queue.cc:1035] T a5bc44bf80f048c3a08e5fb476f7fbbc P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.059830 29929 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:47.060127 29929 tablet.cc:1620] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:47.060222 29636 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:47.060497 29636 tablet.cc:1620] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:47.062979 29930 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:47.063261 29930 tablet.cc:1620] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:47.065560 26506 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=627d54d31b6a427382e9427eba13a774] alter complete (version 2)
I20240502 01:58:47.074813 27064 consensus_queue.cc:237] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.076037 26658 raft_consensus.cc:1261] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.076117 26793 raft_consensus.cc:1261] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:47.076133 26995 consensus_peers.cc:485] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c187cf1abe124e73badcb06ca1a489c4. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.076480 29931 consensus_queue.cc:1035] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.076671 29625 consensus_queue.cc:1035] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.078269 29931 raft_consensus.cc:2932] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.078660 26793 raft_consensus.cc:2932] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.081125 26513 catalog_manager.cc:5484] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.081874 26658 raft_consensus.cc:2932] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.097931 26793 consensus_queue.cc:237] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.099187 26927 raft_consensus.cc:1261] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:47.099352 26730 consensus_peers.cc:485] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6e9b0960a2a94c3fb90902c126ccd034. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.099352 26658 raft_consensus.cc:1261] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.099543 29826 consensus_queue.cc:1035] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.099684 29564 consensus_queue.cc:1035] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.101518 29564 raft_consensus.cc:2932] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.101891 26927 raft_consensus.cc:2932] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.103746 26658 raft_consensus.cc:2932] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.104138 26510 catalog_manager.cc:5484] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.121225 26927 consensus_queue.cc:237] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.122498 26793 raft_consensus.cc:1261] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.122675 27064 raft_consensus.cc:1261] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.122848 28924 consensus_queue.cc:1035] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.123018 29565 consensus_queue.cc:1035] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:47.126242 26859 consensus_peers.cc:485] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 039270108deb4f55b8458ceccf31daa6. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.126770 29565 raft_consensus.cc:2932] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.127240 27064 raft_consensus.cc:2932] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.130105 26793 raft_consensus.cc:2932] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.130381 26506 catalog_manager.cc:5484] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.148445 26793 consensus_queue.cc:237] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.149535 26927 raft_consensus.cc:1261] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.150044 28733 consensus_queue.cc:1035] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.150087 26658 raft_consensus.cc:1261] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:47.150210 26729 consensus_peers.cc:485] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3e2d17af0194d5d8c08ad8372b1e9c1. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.150393 29851 consensus_queue.cc:1035] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.151921 28733 raft_consensus.cc:2932] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.153446 26927 raft_consensus.cc:2932] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.153530 26658 raft_consensus.cc:2932] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.156450 26513 catalog_manager.cc:5484] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.164470 30099 ts_tablet_manager.cc:902] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.165220 30099 tablet_copy_client.cc:308] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.166721 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:47.168496 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:47.172998 29929 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:47.173378 29930 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:47.175608 29636 tablet.cc:1722] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:47.176259 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet b265eec55ee1406b9ff99c159304fa65 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:58:47.176385 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet b265eec55ee1406b9ff99c159304fa65 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-b265eec55ee1406b9ff99c159304fa65
I20240502 01:58:47.177480 26815 tablet_copy_source_session.cc:205] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.179049 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b265eec55ee1406b9ff99c159304fa65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.179692 29329 tablet_service.cc:1467] Processing CreateTablet for tablet fa9fce27ea7c427aa10d13a7528e793b (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.179950 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9fce27ea7c427aa10d13a7528e793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.179955 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 8e66776cedd8407dad9c187b0b04f073 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.180189 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e66776cedd8407dad9c187b0b04f073. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.181216 30099 tablet_copy_client.cc:783] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.181363 30099 tablet_copy_client.cc:647] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.181700 29558 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.182077 30109 tablet_bootstrap.cc:492] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.182292 29330 tablet_service.cc:1467] Processing CreateTablet for tablet ee632f93fe234584814e6aa7b6a63bc0 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.182516 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee632f93fe234584814e6aa7b6a63bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.182534 29558 tablet_bootstrap.cc:654] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.182925 30109 tablet_bootstrap.cc:654] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.183184 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=627d54d31b6a427382e9427eba13a774] alter complete (version 3)
I20240502 01:58:47.184206 30109 tablet_bootstrap.cc:492] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:47.184283 30109 ts_tablet_manager.cc:1367] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:47.184326 26641 tablet_service.cc:1467] Processing CreateTablet for tablet fa9fce27ea7c427aa10d13a7528e793b (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.184567 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9fce27ea7c427aa10d13a7528e793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.184595 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 8e66776cedd8407dad9c187b0b04f073 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.184685 29558 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:47.184754 29558 ts_tablet_manager.cc:1367] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:47.184794 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e66776cedd8407dad9c187b0b04f073. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.184742 30109 raft_consensus.cc:348] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.184831 30109 raft_consensus.cc:374] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.184862 30109 raft_consensus.cc:729] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.184959 30109 consensus_queue.cc:260] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.185142 29558 raft_consensus.cc:348] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.185206 30109 ts_tablet_manager.cc:1398] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.185228 29558 raft_consensus.cc:374] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.185256 29558 raft_consensus.cc:729] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.185367 30109 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.185366 29558 consensus_queue.cc:260] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.185626 29558 ts_tablet_manager.cc:1398] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.186699 29451 tablet_service.cc:1467] Processing CreateTablet for tablet fa9fce27ea7c427aa10d13a7528e793b (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.186861 29195 tablet_service.cc:1467] Processing CreateTablet for tablet ee632f93fe234584814e6aa7b6a63bc0 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.187006 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9fce27ea7c427aa10d13a7528e793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.187063 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee632f93fe234584814e6aa7b6a63bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.188571 29786 tablet_bootstrap.cc:492] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.188874 29454 tablet_service.cc:1467] Processing CreateTablet for tablet ee632f93fe234584814e6aa7b6a63bc0 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.189077 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee632f93fe234584814e6aa7b6a63bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.189450 29786 tablet_bootstrap.cc:654] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.189769 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 8e66776cedd8407dad9c187b0b04f073 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:47.190009 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e66776cedd8407dad9c187b0b04f073. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.191319 30117 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.192410 30117 tablet_bootstrap.cc:654] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.193089 30118 tablet_bootstrap.cc:492] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:47.193540 30120 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:47.194109 30099 tablet_copy_client.cc:523] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.194214 30118 tablet_bootstrap.cc:654] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.194422 30120 tablet_bootstrap.cc:654] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.195297 30109 tablet_bootstrap.cc:654] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.195585 30099 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.196683 30117 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:47.196790 30117 ts_tablet_manager.cc:1367] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:47.197188 29786 tablet_bootstrap.cc:492] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:47.197273 29786 ts_tablet_manager.cc:1367] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:47.197232 30117 raft_consensus.cc:348] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.197572 30117 raft_consensus.cc:374] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.197611 30117 raft_consensus.cc:729] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.197722 30117 consensus_queue.cc:260] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.197757 29786 raft_consensus.cc:348] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.197757 27064 consensus_queue.cc:237] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.197856 29786 raft_consensus.cc:374] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.197890 29786 raft_consensus.cc:729] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.198064 30117 ts_tablet_manager.cc:1398] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.197988 29786 consensus_queue.cc:260] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.198200 30117 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.198267 29786 ts_tablet_manager.cc:1398] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.198418 29786 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.199211 30117 tablet_bootstrap.cc:654] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.199461 29786 tablet_bootstrap.cc:654] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.199488 26927 raft_consensus.cc:1261] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.200012 26793 raft_consensus.cc:1261] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.200465 29931 consensus_queue.cc:1035] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.200774 29931 consensus_queue.cc:1035] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:47.201421 26995 consensus_peers.cc:485] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae9f78db63b74beca267ff5f12a211ae. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.202307 30109 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:47.202438 30109 ts_tablet_manager.cc:1367] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:47.203439 30109 raft_consensus.cc:348] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.203552 30109 raft_consensus.cc:374] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.203593 30109 raft_consensus.cc:729] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.203706 30109 consensus_queue.cc:260] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.203972 30109 ts_tablet_manager.cc:1398] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.204249 28915 raft_consensus.cc:2932] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.204579 29786 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:47.204653 29786 ts_tablet_manager.cc:1367] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:47.205071 29786 raft_consensus.cc:348] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.205159 29786 raft_consensus.cc:374] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.205194 29786 raft_consensus.cc:729] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.205292 29786 consensus_queue.cc:260] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.205534 29786 ts_tablet_manager.cc:1398] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.206867 30120 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:47.206952 30120 ts_tablet_manager.cc:1367] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240502 01:58:47.207376 30120 raft_consensus.cc:348] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.207320 26512 catalog_manager.cc:5484] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.207458 30120 raft_consensus.cc:374] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.207484 30120 raft_consensus.cc:729] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.207585 30120 consensus_queue.cc:260] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.207850 30120 ts_tablet_manager.cc:1398] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:47.209291 26927 raft_consensus.cc:2932] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.210009 30118 tablet_bootstrap.cc:492] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:47.210117 30118 ts_tablet_manager.cc:1367] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 01:58:47.211058 26793 raft_consensus.cc:2932] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.210880 30118 raft_consensus.cc:348] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.212126 30118 raft_consensus.cc:374] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.212170 30118 raft_consensus.cc:729] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.212277 30118 consensus_queue.cc:260] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.212616 30118 ts_tablet_manager.cc:1398] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:47.213228 30117 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:47.213333 30117 ts_tablet_manager.cc:1367] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:58:47.214494 30117 raft_consensus.cc:348] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.214649 30117 raft_consensus.cc:374] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.214727 30117 raft_consensus.cc:729] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.214926 30117 consensus_queue.cc:260] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.216904 30117 ts_tablet_manager.cc:1398] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:47.227775 30099 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.228041 30099 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:47.228232 30099 ts_tablet_manager.cc:1367] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.033s user 0.026s sys 0.000s
I20240502 01:58:47.228542 30099 raft_consensus.cc:348] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.228624 30099 raft_consensus.cc:729] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.228729 30099 consensus_queue.cc:260] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.228989 30099 ts_tablet_manager.cc:1398] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:58:47.229593 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-b265eec55ee1406b9ff99c159304fa65 received from {username='slave'} at 127.19.43.198:56195
I20240502 01:58:47.229676 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-b265eec55ee1406b9ff99c159304fa65 on tablet b265eec55ee1406b9ff99c159304fa65 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:47.234457 27064 consensus_queue.cc:237] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.235750 26661 raft_consensus.cc:1261] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:47.235806 26995 consensus_peers.cc:485] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1417ea39bb2b4716bfb3d11c05853c03. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.236002 26927 raft_consensus.cc:1261] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.236129 29806 consensus_queue.cc:1035] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.236358 29931 consensus_queue.cc:1035] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.237789 29931 raft_consensus.cc:2932] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.238233 26661 raft_consensus.cc:2932] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.240391 26927 raft_consensus.cc:2932] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.240998 26510 catalog_manager.cc:5484] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.258127 26661 consensus_queue.cc:237] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.259949 26927 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.259977 27064 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.260319 28920 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.260617 28920 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.262429 28920 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.262806 27064 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.264484 26927 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:47.265167 26513 catalog_manager.cc:5484] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
W20240502 01:58:47.266443 26595 consensus_peers.cc:485] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1cd4681de8645ff92b479ccdd3eac37. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.271162 30167 raft_consensus.cc:482] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:47.271238 30167 raft_consensus.cc:504] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.271587 30167 leader_election.cc:288] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:47.271855 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9fce27ea7c427aa10d13a7528e793b" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:47.271983 29489 raft_consensus.cc:2454] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:47.272735 26595 leader_election.cc:302] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:47.272888 30167 raft_consensus.cc:2781] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:47.272933 30167 raft_consensus.cc:482] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:47.272965 30167 raft_consensus.cc:3037] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.273747 30167 raft_consensus.cc:504] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.274104 30167 leader_election.cc:288] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:47.274376 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9fce27ea7c427aa10d13a7528e793b" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:47.274477 29489 raft_consensus.cc:3037] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.275271 29489 raft_consensus.cc:2454] T fa9fce27ea7c427aa10d13a7528e793b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:47.275414 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9fce27ea7c427aa10d13a7528e793b" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:47.275487 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9fce27ea7c427aa10d13a7528e793b" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:47.275543 29350 raft_consensus.cc:2454] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:47.275755 26595 leader_election.cc:302] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:47.275988 30167 raft_consensus.cc:2781] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:47.276065 30167 raft_consensus.cc:686] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:47.276171 30167 consensus_queue.cc:237] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:47.279265 26507 catalog_manager.cc:5484] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:47.288787 27064 consensus_queue.cc:237] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.289942 26661 raft_consensus.cc:1261] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:47.290143 26997 consensus_peers.cc:485] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e60816eacfb547cda5a6c87c68bf101d. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.290166 26930 raft_consensus.cc:1261] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.290583 29806 consensus_queue.cc:1035] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.290917 26507 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:47.291016 26507 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:47.292629 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 2c40d8b6702b4265a71986b0102b6268
I20240502 01:58:47.292712 29931 consensus_queue.cc:1035] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.293169 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c40d8b6702b4265a71986b0102b6268 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:47.293226 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c40d8b6702b4265a71986b0102b6268 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:47.293378 29702 tablet_replica.cc:330] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:47.293501 29702 raft_consensus.cc:2229] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:47.293668 29702 raft_consensus.cc:2258] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.293622 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c40d8b6702b4265a71986b0102b6268 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:47.294205 29702 ts_tablet_manager.cc:1875] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.294920 30189 tablet_replica.cc:330] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:47.295359 30189 raft_consensus.cc:2229] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.295864 30189 raft_consensus.cc:2258] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.296025 29702 ts_tablet_manager.cc:1888] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.296089 29702 log.cc:1196] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/2c40d8b6702b4265a71986b0102b6268
I20240502 01:58:47.296283 29702 ts_tablet_manager.cc:1909] T 2c40d8b6702b4265a71986b0102b6268 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:47.296475 30189 ts_tablet_manager.cc:1875] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.296833 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 2c40d8b6702b4265a71986b0102b6268 (table rebalancer_dynamic_table_new_name [id=627d54d31b6a427382e9427eba13a774]) successfully deleted
I20240502 01:58:47.298220 30191 tablet_replica.cc:330] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:47.298485 30191 raft_consensus.cc:2229] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.298671 30191 raft_consensus.cc:2258] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.299554 29806 raft_consensus.cc:2932] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.299672 30191 ts_tablet_manager.cc:1875] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.301259 26661 raft_consensus.cc:2932] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.301990 26507 catalog_manager.cc:5484] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.302835 26930 raft_consensus.cc:2932] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.303855 30191 ts_tablet_manager.cc:1888] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.303934 30191 log.cc:1196] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/2c40d8b6702b4265a71986b0102b6268
I20240502 01:58:47.305111 30191 ts_tablet_manager.cc:1909] T 2c40d8b6702b4265a71986b0102b6268 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:47.305701 30189 ts_tablet_manager.cc:1888] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.306586 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 2c40d8b6702b4265a71986b0102b6268 (table rebalancer_dynamic_table_new_name [id=627d54d31b6a427382e9427eba13a774]) successfully deleted
I20240502 01:58:47.307101 30189 log.cc:1196] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/2c40d8b6702b4265a71986b0102b6268
I20240502 01:58:47.307488 30189 ts_tablet_manager.cc:1909] T 2c40d8b6702b4265a71986b0102b6268 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:47.308135 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 2c40d8b6702b4265a71986b0102b6268 (table rebalancer_dynamic_table_new_name [id=627d54d31b6a427382e9427eba13a774]) successfully deleted
I20240502 01:58:47.323256 26793 consensus_queue.cc:237] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.324573 27064 raft_consensus.cc:1261] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.324717 26661 raft_consensus.cc:1261] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:47.324923 28733 consensus_queue.cc:1035] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:47.325086 29851 consensus_queue.cc:1035] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:47.325416 26727 consensus_peers.cc:485] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 507cc922d2884f48b0f4c43185b4225d. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:47.326905 29851 raft_consensus.cc:2932] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.327267 27064 raft_consensus.cc:2932] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.328570 26661 raft_consensus.cc:2932] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.329730 26510 catalog_manager.cc:5484] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:47.358191 30228 ts_tablet_manager.cc:902] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.358893 30228 tablet_copy_client.cc:308] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.362984 27084 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet b5439ff11ca04925bf46e736a5a53487 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:53349)
I20240502 01:58:47.363116 27084 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet b5439ff11ca04925bf46e736a5a53487 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:53349: session id = 7c693d0933c64d9babc07f404df45cd1-b5439ff11ca04925bf46e736a5a53487
I20240502 01:58:47.364410 27084 tablet_copy_source_session.cc:205] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.366907 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5439ff11ca04925bf46e736a5a53487. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.369146 30228 tablet_copy_client.cc:783] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.373916 30099 ts_tablet_manager.cc:902] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.373975 30259 ts_tablet_manager.cc:902] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.374476 30099 tablet_copy_client.cc:308] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.374522 30259 tablet_copy_client.cc:308] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.375020 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 1c002ce2d74547399685b69a0a4cbada from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:58:47.375110 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 1c002ce2d74547399685b69a0a4cbada from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-1c002ce2d74547399685b69a0a4cbada
I20240502 01:58:47.376022 26815 tablet_copy_source_session.cc:205] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.377305 30228 tablet_copy_client.cc:647] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.377535 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c002ce2d74547399685b69a0a4cbada. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.379230 30228 tablet_copy_client.cc:523] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.379621 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:47995)
I20240502 01:58:47.379658 30099 tablet_copy_client.cc:783] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.379701 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:47995: session id = 7c693d0933c64d9babc07f404df45cd1-6e2ebe00a52240cb9ac5fc8fea9d7151
I20240502 01:58:47.379796 30099 tablet_copy_client.cc:647] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.380450 26815 tablet_copy_source_session.cc:205] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.381254 30228 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.382915 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.385480 30099 tablet_copy_client.cc:523] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.385603 30259 tablet_copy_client.cc:783] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.386694 30099 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.385739 30259 tablet_copy_client.cc:647] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.390925 30259 tablet_copy_client.cc:523] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.391824 30259 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.399412 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:47.404836 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 708088176cfa412a8246fd55f2fb9088 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]), partition=
I20240502 01:58:47.404820 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 708088176cfa412a8246fd55f2fb9088 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]), partition=
I20240502 01:58:47.405021 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 708088176cfa412a8246fd55f2fb9088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.405035 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 708088176cfa412a8246fd55f2fb9088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.405225 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 708088176cfa412a8246fd55f2fb9088 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]), partition=
I20240502 01:58:47.405432 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 708088176cfa412a8246fd55f2fb9088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.406888 29786 tablet_bootstrap.cc:492] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.407477 30109 tablet_bootstrap.cc:492] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.407893 29786 tablet_bootstrap.cc:654] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.408288 30109 tablet_bootstrap.cc:654] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.409175 29558 tablet_bootstrap.cc:492] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.409489 30109 tablet_bootstrap.cc:492] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:47.409564 30109 ts_tablet_manager.cc:1367] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:47.409891 29558 tablet_bootstrap.cc:654] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:47.410061 30109 raft_consensus.cc:348] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.410161 30109 raft_consensus.cc:374] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.410192 30109 raft_consensus.cc:729] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.410239 29786 tablet_bootstrap.cc:492] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:47.410310 29786 ts_tablet_manager.cc:1367] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:47.410306 30109 consensus_queue.cc:260] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.410578 30109 ts_tablet_manager.cc:1398] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.410715 29786 raft_consensus.cc:348] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.410825 29786 raft_consensus.cc:374] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.410861 29786 raft_consensus.cc:729] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.410977 29786 consensus_queue.cc:260] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.411041 29992 raft_consensus.cc:482] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:47.411099 29992 raft_consensus.cc:504] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.411217 29786 ts_tablet_manager.cc:1398] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.411531 29992 leader_election.cc:288] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:47.417675 29558 tablet_bootstrap.cc:492] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:47.417765 29558 ts_tablet_manager.cc:1367] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:47.418183 29558 raft_consensus.cc:348] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.418272 29558 raft_consensus.cc:374] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:47.418295 29558 raft_consensus.cc:729] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.418392 29558 consensus_queue.cc:260] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.418606 29558 ts_tablet_manager.cc:1398] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.423044 26507 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 01:58:47.423157 26507 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 01:58:47.424253 29788 raft_consensus.cc:482] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:47.424345 29788 raft_consensus.cc:504] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.425328 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet ce3d632b976e491e8ac726952c69cbc3
I20240502 01:58:47.425423 29788 leader_election.cc:288] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:47.425629 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet c0d1a201768f4f68b3bca4197d6d15c2
I20240502 01:58:47.425856 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet e812884ef6474b5384d026637dc62b61
I20240502 01:58:47.426848 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3d632b976e491e8ac726952c69cbc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:47.426848 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet e812884ef6474b5384d026637dc62b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:47.427042 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d1a201768f4f68b3bca4197d6d15c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:47.427232 29703 tablet_replica.cc:330] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:47.427371 29703 raft_consensus.cc:2229] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.427529 29703 raft_consensus.cc:2258] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.427892 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3d632b976e491e8ac726952c69cbc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:47.428045 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d1a201768f4f68b3bca4197d6d15c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:47.428133 29705 tablet_replica.cc:330] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:47.428278 29705 raft_consensus.cc:2229] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:47.428539 29705 raft_consensus.cc:2258] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.428823 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3d632b976e491e8ac726952c69cbc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:47.429237 29705 ts_tablet_manager.cc:1875] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.429656 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet e812884ef6474b5384d026637dc62b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:47.430011 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3d632b976e491e8ac726952c69cbc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:47.430204 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet e812884ef6474b5384d026637dc62b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:47.430251 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet e812884ef6474b5384d026637dc62b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:47.430347 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d1a201768f4f68b3bca4197d6d15c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:47.430506 29702 tablet_replica.cc:330] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:47.430644 29702 raft_consensus.cc:2229] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.430806 29702 raft_consensus.cc:2258] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.430828 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:47.430930 29350 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:47.430984 29704 tablet_replica.cc:330] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:47.431123 29704 raft_consensus.cc:2229] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:47.431393 29704 raft_consensus.cc:2258] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.431468 29705 ts_tablet_manager.cc:1888] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.431529 29705 log.cc:1196] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/ce3d632b976e491e8ac726952c69cbc3
I20240502 01:58:47.431898 29419 leader_election.cc:302] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:47.432001 29703 ts_tablet_manager.cc:1875] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.432142 29788 raft_consensus.cc:2781] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:47.432193 29788 raft_consensus.cc:482] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:47.432209 29702 ts_tablet_manager.cc:1875] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.432227 29788 raft_consensus.cc:3037] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.432940 29705 ts_tablet_manager.cc:1909] T ce3d632b976e491e8ac726952c69cbc3 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:47.433085 29788 raft_consensus.cc:504] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.433388 29788 leader_election.cc:288] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:47.433542 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet ce3d632b976e491e8ac726952c69cbc3 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.433740 29705 tablet_replica.cc:330] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:47.433773 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:47.433856 29349 raft_consensus.cc:3037] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.433859 29705 raft_consensus.cc:2229] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:47.434012 29705 raft_consensus.cc:2258] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.434224 29702 ts_tablet_manager.cc:1888] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.434284 29702 log.cc:1196] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce3d632b976e491e8ac726952c69cbc3
I20240502 01:58:47.434475 29705 ts_tablet_manager.cc:1875] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.434546 29702 ts_tablet_manager.cc:1909] T ce3d632b976e491e8ac726952c69cbc3 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:47.434762 29349 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:47.435026 29419 leader_election.cc:302] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:47.435053 29702 tablet_replica.cc:330] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:47.435174 29702 raft_consensus.cc:2229] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.435235 29788 raft_consensus.cc:2781] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:47.435256 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet ce3d632b976e491e8ac726952c69cbc3 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.435302 29702 raft_consensus.cc:2258] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.435410 29788 raft_consensus.cc:686] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
W20240502 01:58:47.435484 26502 catalog_manager.cc:4797] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): delete failed for tablet e812884ef6474b5384d026637dc62b61 because the tablet was not found. No further retry: Not found: Tablet not found: e812884ef6474b5384d026637dc62b61
I20240502 01:58:47.435618 29788 consensus_queue.cc:237] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.436215 29704 ts_tablet_manager.cc:1875] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.436254 29703 ts_tablet_manager.cc:1888] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.436275 29705 ts_tablet_manager.cc:1888] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:47.436316 29703 log.cc:1196] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/ce3d632b976e491e8ac726952c69cbc3
I20240502 01:58:47.436333 29705 log.cc:1196] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/c0d1a201768f4f68b3bca4197d6d15c2
I20240502 01:58:47.436547 29703 ts_tablet_manager.cc:1909] T ce3d632b976e491e8ac726952c69cbc3 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:47.436614 29705 ts_tablet_manager.cc:1909] T c0d1a201768f4f68b3bca4197d6d15c2 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:47.437148 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet ce3d632b976e491e8ac726952c69cbc3 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.437346 29703 tablet_replica.cc:330] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:47.437467 29703 raft_consensus.cc:2229] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.437580 29703 raft_consensus.cc:2258] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.438107 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet c0d1a201768f4f68b3bca4197d6d15c2 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.438248 29703 ts_tablet_manager.cc:1875] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.439991 29703 ts_tablet_manager.cc:1888] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.440047 29703 log.cc:1196] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/e812884ef6474b5384d026637dc62b61
I20240502 01:58:47.440265 29703 ts_tablet_manager.cc:1909] T e812884ef6474b5384d026637dc62b61 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:47.440913 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet e812884ef6474b5384d026637dc62b61 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.440946 26507 catalog_manager.cc:5484] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.441095 29703 tablet_replica.cc:330] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:47.441205 29703 raft_consensus.cc:2229] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.441306 29703 raft_consensus.cc:2258] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.441864 29704 ts_tablet_manager.cc:1888] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.441928 29704 log.cc:1196] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/e812884ef6474b5384d026637dc62b61
I20240502 01:58:47.442457 29704 ts_tablet_manager.cc:1909] T e812884ef6474b5384d026637dc62b61 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:47.442504 30282 ts_tablet_manager.cc:902] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.442728 30283 ts_tablet_manager.cc:902] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.443038 29704 tablet_replica.cc:330] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:47.443209 29704 raft_consensus.cc:2229] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.443215 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet e812884ef6474b5384d026637dc62b61 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.443390 30283 tablet_copy_client.cc:308] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.443603 29704 raft_consensus.cc:2258] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.444279 29703 ts_tablet_manager.cc:1875] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.444729 30282 tablet_copy_client.cc:308] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.445497 29704 ts_tablet_manager.cc:1875] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.446576 29997 raft_consensus.cc:482] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:47.446660 29997 raft_consensus.cc:504] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.446935 29703 ts_tablet_manager.cc:1888] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:47.446993 29703 log.cc:1196] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0d1a201768f4f68b3bca4197d6d15c2
I20240502 01:58:47.447088 29997 leader_election.cc:288] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:47.447244 29703 ts_tablet_manager.cc:1909] T c0d1a201768f4f68b3bca4197d6d15c2 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:47.447780 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet c0d1a201768f4f68b3bca4197d6d15c2 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.449240 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:47.449946 29704 ts_tablet_manager.cc:1888] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:47.450090 29704 log.cc:1196] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/c0d1a201768f4f68b3bca4197d6d15c2
I20240502 01:58:47.450460 29704 ts_tablet_manager.cc:1909] T c0d1a201768f4f68b3bca4197d6d15c2 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:47.450920 29702 ts_tablet_manager.cc:1875] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.451335 30099 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.451578 30099 tablet_bootstrap.cc:492] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:47.451730 30099 ts_tablet_manager.cc:1367] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.065s user 0.025s sys 0.000s
I20240502 01:58:47.452170 30099 raft_consensus.cc:348] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.452299 30099 raft_consensus.cc:729] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.452451 30099 consensus_queue.cc:260] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.453140 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 565aab51035c4f328e49eca53eddf621 from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:53523)
I20240502 01:58:47.453234 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 565aab51035c4f328e49eca53eddf621 from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:53523: session id = f3d18a019013487ab801599cfb80e4b4-565aab51035c4f328e49eca53eddf621
I20240502 01:58:47.453260 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e66776cedd8407dad9c187b0b04f073" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:47.453361 26794 raft_consensus.cc:2454] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:47.453461 29702 ts_tablet_manager.cc:1888] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.453583 29702 log.cc:1196] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/e812884ef6474b5384d026637dc62b61
I20240502 01:58:47.453876 29702 ts_tablet_manager.cc:1909] T e812884ef6474b5384d026637dc62b61 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:47.454123 26815 tablet_copy_source_session.cc:205] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.454146 30278 ts_tablet_manager.cc:902] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.454479 30099 ts_tablet_manager.cc:1398] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 01:58:47.454550 29015 leader_election.cc:302] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:47.454622 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet e812884ef6474b5384d026637dc62b61 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.454865 30278 tablet_copy_client.cc:308] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.455036 29997 raft_consensus.cc:2781] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:47.455088 29997 raft_consensus.cc:482] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:47.455123 29997 raft_consensus.cc:3037] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.456046 29997 raft_consensus.cc:504] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.456315 29997 leader_election.cc:288] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:47.456573 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e66776cedd8407dad9c187b0b04f073" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:47.456645 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-1c002ce2d74547399685b69a0a4cbada received from {username='slave'} at 127.19.43.198:56195
I20240502 01:58:47.456671 26794 raft_consensus.cc:3037] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.456702 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-1c002ce2d74547399685b69a0a4cbada on tablet 1c002ce2d74547399685b69a0a4cbada with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:47.457515 26794 raft_consensus.cc:2454] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:47.457564 30283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565aab51035c4f328e49eca53eddf621. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.457793 29015 leader_election.cc:302] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:47.457938 29997 raft_consensus.cc:2781] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:47.458009 29997 raft_consensus.cc:686] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:47.458138 29997 consensus_queue.cc:237] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.459651 26509 catalog_manager.cc:5484] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:47.460827 30283 tablet_copy_client.cc:783] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.461030 30283 tablet_copy_client.cc:647] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.453934 27084 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet 7f5513cf299e44038525ad5f438d74ec from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:56105)
I20240502 01:58:47.462361 27084 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet 7f5513cf299e44038525ad5f438d74ec from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:56105: session id = 468dd27456c64d67b22e8a58af93d93d-7f5513cf299e44038525ad5f438d74ec
I20240502 01:58:47.463256 30283 tablet_copy_client.cc:523] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.463254 27084 tablet_copy_source_session.cc:205] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.455332 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet cceb8a91997e4db9a4842df26ad3b02e from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:56105)
I20240502 01:58:47.463536 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet cceb8a91997e4db9a4842df26ad3b02e from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:56105: session id = 468dd27456c64d67b22e8a58af93d93d-cceb8a91997e4db9a4842df26ad3b02e
I20240502 01:58:47.463627 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:47.463733 29216 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:47.464010 29285 leader_election.cc:302] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:47.464291 30274 raft_consensus.cc:2741] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20240502 01:58:47.464320 27083 tablet_copy_source_session.cc:205] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.464570 30283 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.465415 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet c0d1a201768f4f68b3bca4197d6d15c2 (table rebalance_test_table_4 [id=d1cbab12dd3d44308b638d046df39658]) successfully deleted
I20240502 01:58:47.465572 30282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f5513cf299e44038525ad5f438d74ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 01:58:47.465709 26499 catalog_manager.cc:4797] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): delete failed for tablet ce3d632b976e491e8ac726952c69cbc3 because the tablet was not found. No further retry: Not found: Tablet not found: ce3d632b976e491e8ac726952c69cbc3
I20240502 01:58:47.467238 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:47.467360 29216 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:47.467486 30278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cceb8a91997e4db9a4842df26ad3b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.467550 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:47.467628 29213 raft_consensus.cc:3037] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.467728 30282 tablet_copy_client.cc:783] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.467891 30282 tablet_copy_client.cc:647] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.468513 29213 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:47.469309 30278 tablet_copy_client.cc:783] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.469436 30278 tablet_copy_client.cc:647] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.470165 30282 tablet_copy_client.cc:523] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.471632 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e66776cedd8407dad9c187b0b04f073" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:47.471737 29213 raft_consensus.cc:2454] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:47.471909 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e66776cedd8407dad9c187b0b04f073" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:47.471992 29216 raft_consensus.cc:3037] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.472046 30282 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.472774 29216 raft_consensus.cc:2454] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:47.480132 30259 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.480438 30259 tablet_bootstrap.cc:492] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:47.480604 30259 ts_tablet_manager.cc:1367] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.089s user 0.024s sys 0.000s
I20240502 01:58:47.481007 30259 raft_consensus.cc:348] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.481117 30259 raft_consensus.cc:729] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.481258 30259 consensus_queue.cc:260] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.482846 30259 ts_tablet_manager.cc:1398] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:47.483835 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-6e2ebe00a52240cb9ac5fc8fea9d7151 received from {username='slave'} at 127.19.43.200:47995
I20240502 01:58:47.483898 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-6e2ebe00a52240cb9ac5fc8fea9d7151 on tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:47.486363 29195 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.488471 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:47.489221 27043 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:58:47.497485 26639 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:47.498647 29058 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:47.502200 30278 tablet_copy_client.cc:523] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.503386 30278 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.503851 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.509069 26907 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:47.511178 30228 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.511425 30228 tablet_bootstrap.cc:492] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:47.511554 30228 ts_tablet_manager.cc:1367] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.130s user 0.026s sys 0.003s
I20240502 01:58:47.516325 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:47.538741 30228 raft_consensus.cc:348] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.538878 30228 raft_consensus.cc:729] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.535679 30318 ts_tablet_manager.cc:902] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.539875 30228 consensus_queue.cc:260] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.544705 30318 tablet_copy_client.cc:308] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.545105 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet c187cf1abe124e73badcb06ca1a489c4 from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:55881)
I20240502 01:58:47.545210 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet c187cf1abe124e73badcb06ca1a489c4 from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:55881: session id = f3d18a019013487ab801599cfb80e4b4-c187cf1abe124e73badcb06ca1a489c4
I20240502 01:58:47.546135 27083 tablet_copy_source_session.cc:205] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.546550 30228 ts_tablet_manager.cc:1398] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.035s user 0.002s sys 0.000s
I20240502 01:58:47.546967 30318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c187cf1abe124e73badcb06ca1a489c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.549371 30318 tablet_copy_client.cc:783] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.549512 30318 tablet_copy_client.cc:647] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.550374 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-b5439ff11ca04925bf46e736a5a53487 received from {username='slave'} at 127.19.43.200:53349
I20240502 01:58:47.550462 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-b5439ff11ca04925bf46e736a5a53487 on tablet b5439ff11ca04925bf46e736a5a53487 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:47.554013 30318 tablet_copy_client.cc:523] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.555599 30099 ts_tablet_manager.cc:902] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.555666 30318 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.556015 30099 tablet_copy_client.cc:308] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.556433 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet cdd32c8294b44540ac5b832cc48d2519 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:58:47.556538 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet cdd32c8294b44540ac5b832cc48d2519 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-cdd32c8294b44540ac5b832cc48d2519
I20240502 01:58:47.557247 30319 ts_tablet_manager.cc:902] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.557386 26815 tablet_copy_source_session.cc:205] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.557911 30319 tablet_copy_client.cc:308] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.558203 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdd32c8294b44540ac5b832cc48d2519. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.558306 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet 936e77ced55c41d6bfe955d9a4271b80 from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:55881)
I20240502 01:58:47.558403 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet 936e77ced55c41d6bfe955d9a4271b80 from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:55881: session id = f3d18a019013487ab801599cfb80e4b4-936e77ced55c41d6bfe955d9a4271b80
I20240502 01:58:47.558573 30278 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.558894 30278 tablet_bootstrap.cc:492] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:47.559051 30278 ts_tablet_manager.cc:1367] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.056s user 0.030s sys 0.000s
I20240502 01:58:47.559264 27083 tablet_copy_source_session.cc:205] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.559496 30278 raft_consensus.cc:348] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.559620 30278 raft_consensus.cc:729] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.559839 30278 consensus_queue.cc:260] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.560390 30099 tablet_copy_client.cc:783] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.560537 30099 tablet_copy_client.cc:647] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.560886 30278 ts_tablet_manager.cc:1398] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:47.561338 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-cceb8a91997e4db9a4842df26ad3b02e received from {username='slave'} at 127.19.43.199:56105
I20240502 01:58:47.561403 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-cceb8a91997e4db9a4842df26ad3b02e on tablet cceb8a91997e4db9a4842df26ad3b02e with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:47.562678 30319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 936e77ced55c41d6bfe955d9a4271b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.564204 30319 tablet_copy_client.cc:783] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.564328 30319 tablet_copy_client.cc:647] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.568192 30319 tablet_copy_client.cc:523] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.569290 30319 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.592464 30099 tablet_copy_client.cc:523] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.593711 30099 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.594163 30318 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.594491 30318 tablet_bootstrap.cc:492] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:47.594641 30318 ts_tablet_manager.cc:1367] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.039s user 0.022s sys 0.003s
I20240502 01:58:47.595058 30318 raft_consensus.cc:348] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.595172 30318 raft_consensus.cc:729] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.595376 30318 consensus_queue.cc:260] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.596179 30318 ts_tablet_manager.cc:1398] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.596657 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-c187cf1abe124e73badcb06ca1a489c4 received from {username='slave'} at 127.19.43.197:55881
I20240502 01:58:47.596735 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-c187cf1abe124e73badcb06ca1a489c4 on tablet c187cf1abe124e73badcb06ca1a489c4 with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:47.597720 30282 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.597975 30282 tablet_bootstrap.cc:492] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:47.598152 30282 ts_tablet_manager.cc:1367] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.126s user 0.023s sys 0.003s
I20240502 01:58:47.600507 30319 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.600731 30319 tablet_bootstrap.cc:492] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:47.600881 30319 ts_tablet_manager.cc:1367] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.004s
I20240502 01:58:47.601279 30319 raft_consensus.cc:348] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.601378 30319 raft_consensus.cc:729] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.601514 30319 consensus_queue.cc:260] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.602309 30319 ts_tablet_manager.cc:1398] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:58:47.602816 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-936e77ced55c41d6bfe955d9a4271b80 received from {username='slave'} at 127.19.43.197:55881
I20240502 01:58:47.602877 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-936e77ced55c41d6bfe955d9a4271b80 on tablet 936e77ced55c41d6bfe955d9a4271b80 with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:47.606509 30282 raft_consensus.cc:348] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.606779 30282 raft_consensus.cc:729] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.606974 30282 consensus_queue.cc:260] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.607825 30282 ts_tablet_manager.cc:1398] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:47.608217 30283 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.608325 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-7f5513cf299e44038525ad5f438d74ec received from {username='slave'} at 127.19.43.199:56105
I20240502 01:58:47.608383 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-7f5513cf299e44038525ad5f438d74ec on tablet 7f5513cf299e44038525ad5f438d74ec with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:47.608453 30283 tablet_bootstrap.cc:492] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:47.608615 30283 ts_tablet_manager.cc:1367] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.144s user 0.026s sys 0.000s
I20240502 01:58:47.608989 30283 raft_consensus.cc:348] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.609090 30283 raft_consensus.cc:729] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.609225 30283 consensus_queue.cc:260] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.609673 30283 ts_tablet_manager.cc:1398] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:58:47.610275 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-565aab51035c4f328e49eca53eddf621 received from {username='slave'} at 127.19.43.197:53523
I20240502 01:58:47.610347 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-565aab51035c4f328e49eca53eddf621 on tablet 565aab51035c4f328e49eca53eddf621 with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:47.623734 30099 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.623968 30099 tablet_bootstrap.cc:492] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:47.624121 30099 ts_tablet_manager.cc:1367] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.030s user 0.018s sys 0.000s
I20240502 01:58:47.624531 30099 raft_consensus.cc:348] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.624640 30099 raft_consensus.cc:729] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.624781 30099 consensus_queue.cc:260] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.625032 30099 ts_tablet_manager.cc:1398] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:58:47.626080 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-cdd32c8294b44540ac5b832cc48d2519 received from {username='slave'} at 127.19.43.198:56195
I20240502 01:58:47.626153 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-cdd32c8294b44540ac5b832cc48d2519 on tablet cdd32c8294b44540ac5b832cc48d2519 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:47.627358 30274 raft_consensus.cc:482] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:47.627450 30274 raft_consensus.cc:504] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.627768 30274 leader_election.cc:288] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:47.628087 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "708088176cfa412a8246fd55f2fb9088" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:47.628211 29080 raft_consensus.cc:2454] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:47.628393 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "708088176cfa412a8246fd55f2fb9088" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:47.628512 29283 leader_election.cc:302] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:47.628526 29489 raft_consensus.cc:2454] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:47.628654 30274 raft_consensus.cc:2781] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:47.628706 30274 raft_consensus.cc:482] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:47.628752 30274 raft_consensus.cc:3037] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.629688 30274 raft_consensus.cc:504] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.630151 30274 leader_election.cc:288] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:47.630419 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "708088176cfa412a8246fd55f2fb9088" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:47.630527 29080 raft_consensus.cc:3037] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.630703 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "708088176cfa412a8246fd55f2fb9088" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:47.630797 29489 raft_consensus.cc:3037] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.631420 29080 raft_consensus.cc:2454] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:47.631642 29489 raft_consensus.cc:2454] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:47.631703 29283 leader_election.cc:302] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:47.631850 30274 raft_consensus.cc:2781] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:47.631906 30274 raft_consensus.cc:686] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:47.632025 30274 consensus_queue.cc:237] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.633584 26509 catalog_manager.cc:5484] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:47.661532 30228 ts_tablet_manager.cc:902] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:47.661958 30228 tablet_copy_client.cc:308] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:58:47.666064 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet e1cd4681de8645ff92b479ccdd3eac37 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:59383)
I20240502 01:58:47.666191 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet e1cd4681de8645ff92b479ccdd3eac37 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:59383: session id = 7c693d0933c64d9babc07f404df45cd1-e1cd4681de8645ff92b479ccdd3eac37
I20240502 01:58:47.667351 26682 tablet_copy_source_session.cc:205] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.668357 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1cd4681de8645ff92b479ccdd3eac37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.669976 30228 tablet_copy_client.cc:783] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.670147 30228 tablet_copy_client.cc:647] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.671595 30228 tablet_copy_client.cc:523] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.672719 30228 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.682917 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.684384 29195 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.686167 30259 ts_tablet_manager.cc:902] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.686602 30259 tablet_copy_client.cc:308] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.687088 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 6e9b0960a2a94c3fb90902c126ccd034 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:47995)
I20240502 01:58:47.687201 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 6e9b0960a2a94c3fb90902c126ccd034 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:47995: session id = 7c693d0933c64d9babc07f404df45cd1-6e9b0960a2a94c3fb90902c126ccd034
I20240502 01:58:47.688042 26815 tablet_copy_source_session.cc:205] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.688730 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.689180 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9b0960a2a94c3fb90902c126ccd034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.691377 30259 tablet_copy_client.cc:783] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.691498 30259 tablet_copy_client.cc:647] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.693276 26907 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:47.698451 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:47.706260 30228 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.706506 30228 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:47.706668 30228 ts_tablet_manager.cc:1367] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.003s
I20240502 01:58:47.707070 30228 raft_consensus.cc:348] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.707207 30228 raft_consensus.cc:729] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.707255 27043 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:58:47.707362 30228 consensus_queue.cc:260] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.707774 30099 ts_tablet_manager.cc:902] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.708175 30099 tablet_copy_client.cc:308] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.708618 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:58:47.708700 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet d3e2d17af0194d5d8c08ad8372b1e9c1 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-d3e2d17af0194d5d8c08ad8372b1e9c1
I20240502 01:58:47.708954 30228 ts_tablet_manager.cc:1398] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:47.709421 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-e1cd4681de8645ff92b479ccdd3eac37 received from {username='slave'} at 127.19.43.200:59383
I20240502 01:58:47.709499 26815 tablet_copy_source_session.cc:205] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.709510 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-e1cd4681de8645ff92b479ccdd3eac37 on tablet e1cd4681de8645ff92b479ccdd3eac37 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:47.710494 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e2d17af0194d5d8c08ad8372b1e9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.712402 30099 tablet_copy_client.cc:783] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.712527 30099 tablet_copy_client.cc:647] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.714119 30166 consensus_queue.cc:1035] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.716303 30099 tablet_copy_client.cc:523] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.717562 30099 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.722245 29349 raft_consensus.cc:3037] T fa9fce27ea7c427aa10d13a7528e793b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:47.723608 30166 consensus_queue.cc:1035] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.725209 30282 ts_tablet_manager.cc:902] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.725550 30282 tablet_copy_client.cc:308] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.726213 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet 1417ea39bb2b4716bfb3d11c05853c03 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:56105)
I20240502 01:58:47.726298 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet 1417ea39bb2b4716bfb3d11c05853c03 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:56105: session id = 468dd27456c64d67b22e8a58af93d93d-1417ea39bb2b4716bfb3d11c05853c03
I20240502 01:58:47.727209 27083 tablet_copy_source_session.cc:205] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.727998 30282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1417ea39bb2b4716bfb3d11c05853c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.729846 30282 tablet_copy_client.cc:783] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.729982 30282 tablet_copy_client.cc:647] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.732126 30278 ts_tablet_manager.cc:902] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:47.735294 30278 tablet_copy_client.cc:308] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:47.736218 26639 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:47.737234 30259 tablet_copy_client.cc:523] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.738368 30259 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:47.743055 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 039270108deb4f55b8458ceccf31daa6 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:43421)
I20240502 01:58:47.743175 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 039270108deb4f55b8458ceccf31daa6 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:43421: session id = 468dd27456c64d67b22e8a58af93d93d-039270108deb4f55b8458ceccf31daa6
I20240502 01:58:47.744433 26950 tablet_copy_source_session.cc:205] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.745458 30278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039270108deb4f55b8458ceccf31daa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.747349 30278 tablet_copy_client.cc:783] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.747473 30278 tablet_copy_client.cc:647] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.747879 30282 tablet_copy_client.cc:523] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.748878 30282 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.753173 30278 tablet_copy_client.cc:523] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.754160 30278 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:47.756850 29058 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:47.773357 29216 raft_consensus.cc:1203] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.773873 30282 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.778398 30282 tablet_bootstrap.cc:492] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:47.778566 30282 ts_tablet_manager.cc:1367] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.030s user 0.017s sys 0.008s
I20240502 01:58:47.778968 30282 raft_consensus.cc:348] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.779090 30282 raft_consensus.cc:729] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.779229 30282 consensus_queue.cc:260] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.779498 30282 ts_tablet_manager.cc:1398] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:47.780227 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-1417ea39bb2b4716bfb3d11c05853c03 received from {username='slave'} at 127.19.43.199:56105
I20240502 01:58:47.780304 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-1417ea39bb2b4716bfb3d11c05853c03 on tablet 1417ea39bb2b4716bfb3d11c05853c03 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:47.781773 30319 ts_tablet_manager.cc:902] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:47.782207 30319 tablet_copy_client.cc:308] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:47.782531 29489 raft_consensus.cc:1203] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.783569 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 507cc922d2884f48b0f4c43185b4225d from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:53523)
I20240502 01:58:47.783664 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 507cc922d2884f48b0f4c43185b4225d from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:53523: session id = f3d18a019013487ab801599cfb80e4b4-507cc922d2884f48b0f4c43185b4225d
I20240502 01:58:47.784507 26815 tablet_copy_source_session.cc:205] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.786020 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:47.787287 30319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507cc922d2884f48b0f4c43185b4225d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.788105 30259 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.788360 30259 tablet_bootstrap.cc:492] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:47.788388 30283 ts_tablet_manager.cc:902] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.788529 30259 ts_tablet_manager.cc:1367] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.050s user 0.016s sys 0.006s
I20240502 01:58:47.788851 30283 tablet_copy_client.cc:308] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.788930 30259 raft_consensus.cc:348] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.789029 30259 raft_consensus.cc:729] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.789140 30259 consensus_queue.cc:260] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:47.789290 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet ae9f78db63b74beca267ff5f12a211ae from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:55881)
I20240502 01:58:47.789378 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet ae9f78db63b74beca267ff5f12a211ae from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:55881: session id = f3d18a019013487ab801599cfb80e4b4-ae9f78db63b74beca267ff5f12a211ae
I20240502 01:58:47.789392 30259 ts_tablet_manager.cc:1398] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:47.790302 27083 tablet_copy_source_session.cc:205] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.790369 30319 tablet_copy_client.cc:783] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.790351 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-6e9b0960a2a94c3fb90902c126ccd034 received from {username='slave'} at 127.19.43.200:47995
I20240502 01:58:47.790417 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-6e9b0960a2a94c3fb90902c126ccd034 on tablet 6e9b0960a2a94c3fb90902c126ccd034 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:47.790493 30319 tablet_copy_client.cc:647] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.792829 29489 raft_consensus.cc:1261] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:47.793515 30283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae9f78db63b74beca267ff5f12a211ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.793756 30099 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.793864 29080 raft_consensus.cc:1261] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:47.793993 30099 tablet_bootstrap.cc:492] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:47.794176 30099 ts_tablet_manager.cc:1367] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.077s user 0.024s sys 0.001s
I20240502 01:58:47.794258 29992 consensus_queue.cc:1035] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.794615 30099 raft_consensus.cc:348] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.794723 30099 raft_consensus.cc:729] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.794816 29992 consensus_queue.cc:1035] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.795416 30283 tablet_copy_client.cc:783] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.795542 30283 tablet_copy_client.cc:647] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.795748 30319 tablet_copy_client.cc:523] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.796797 30319 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.797217 26513 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]). Expected version 1 got 0
I20240502 01:58:47.799407 26513 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]). Expected version 1 got 0
I20240502 01:58:47.799691 30099 consensus_queue.cc:260] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.803261 30419 ts_tablet_manager.cc:902] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:47.803846 30419 tablet_copy_client.cc:308] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:47.804028 30405 mvcc.cc:205] Tried to move back new op lower bound from 7023063563432210432 to 7023063562782113792. Current Snapshot: MvccSnapshot[applied={T|T < 7023063563432210432}]
I20240502 01:58:47.804760 30283 tablet_copy_client.cc:523] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.804797 30443 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:47.805244 30443 tablet.cc:1620] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:47.805804 30283 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:47.805853 26513 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]). Expected version 1 got 0
I20240502 01:58:47.810427 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745] alter complete (version 1)
I20240502 01:58:47.818013 30278 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.818348 30099 ts_tablet_manager.cc:1398] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.024s user 0.000s sys 0.001s
I20240502 01:58:47.818851 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-d3e2d17af0194d5d8c08ad8372b1e9c1 received from {username='slave'} at 127.19.43.198:56195
I20240502 01:58:47.818922 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-d3e2d17af0194d5d8c08ad8372b1e9c1 on tablet d3e2d17af0194d5d8c08ad8372b1e9c1 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:47.819979 30278 tablet_bootstrap.cc:492] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:47.820135 30278 ts_tablet_manager.cc:1367] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.066s user 0.018s sys 0.003s
I20240502 01:58:47.821264 30278 raft_consensus.cc:348] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.821355 30278 raft_consensus.cc:729] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.821491 30278 consensus_queue.cc:260] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:47.821863 30278 ts_tablet_manager.cc:1398] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 01:58:47.822410 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-039270108deb4f55b8458ceccf31daa6 received from {username='slave'} at 127.19.43.199:43421
I20240502 01:58:47.822489 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-039270108deb4f55b8458ceccf31daa6 on tablet 039270108deb4f55b8458ceccf31daa6 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:47.825526 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet e60816eacfb547cda5a6c87c68bf101d from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:58871)
I20240502 01:58:47.825618 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet e60816eacfb547cda5a6c87c68bf101d from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:58871: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-e60816eacfb547cda5a6c87c68bf101d
I20240502 01:58:47.826603 27083 tablet_copy_source_session.cc:205] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:47.827829 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e60816eacfb547cda5a6c87c68bf101d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:47.828387 30399 mvcc.cc:205] Tried to move back new op lower bound from 7023063563432210432 to 7023063562782113792. Current Snapshot: MvccSnapshot[applied={T|T < 7023063563432210432}]
I20240502 01:58:47.829517 26513 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745]). Expected version 1 got 0
I20240502 01:58:47.829885 30419 tablet_copy_client.cc:783] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:47.830080 30419 tablet_copy_client.cc:647] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:47.830159 30440 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:47.830426 30440 tablet.cc:1620] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:47.831089 30447 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:47.831432 30447 tablet.cc:1620] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:47.839387 30419 tablet_copy_client.cc:523] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:47.840708 30419 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:47.843123 30283 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.843420 30283 tablet_bootstrap.cc:492] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:47.843564 30283 ts_tablet_manager.cc:1367] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.038s user 0.025s sys 0.000s
I20240502 01:58:47.843958 30283 raft_consensus.cc:348] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.844055 30283 raft_consensus.cc:729] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.844206 30283 consensus_queue.cc:260] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.844542 30283 ts_tablet_manager.cc:1398] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:47.844923 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-ae9f78db63b74beca267ff5f12a211ae received from {username='slave'} at 127.19.43.197:55881
I20240502 01:58:47.844985 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-ae9f78db63b74beca267ff5f12a211ae on tablet ae9f78db63b74beca267ff5f12a211ae with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:47.845973 30319 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.846220 30319 tablet_bootstrap.cc:492] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:47.846343 30319 ts_tablet_manager.cc:1367] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.050s user 0.025s sys 0.000s
I20240502 01:58:47.846685 30319 raft_consensus.cc:348] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.846777 30319 raft_consensus.cc:729] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.846879 30319 consensus_queue.cc:260] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:47.847090 30319 ts_tablet_manager.cc:1398] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:47.847465 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-507cc922d2884f48b0f4c43185b4225d received from {username='slave'} at 127.19.43.197:53523
I20240502 01:58:47.847527 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-507cc922d2884f48b0f4c43185b4225d on tablet 507cc922d2884f48b0f4c43185b4225d with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:47.859751 29489 raft_consensus.cc:1203] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.860558 29349 raft_consensus.cc:1203] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.864262 29080 raft_consensus.cc:1203] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.874516 30419 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:47.875169 30284 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.875770 30419 tablet_bootstrap.cc:492] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:47.875993 30419 ts_tablet_manager.cc:1367] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.035s user 0.022s sys 0.001s
I20240502 01:58:47.876462 30419 raft_consensus.cc:348] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.876631 30419 raft_consensus.cc:729] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:47.876863 30419 consensus_queue.cc:260] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:47.877141 30419 ts_tablet_manager.cc:1398] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:47.877596 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-e60816eacfb547cda5a6c87c68bf101d received from {username='slave'} at 127.19.43.198:58871
I20240502 01:58:47.877666 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-e60816eacfb547cda5a6c87c68bf101d on tablet e60816eacfb547cda5a6c87c68bf101d with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:47.882151 29788 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.915180 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:47.916235 29195 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.920487 29080 raft_consensus.cc:1203] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.921830 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:47.922809 29216 raft_consensus.cc:1203] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.928107 30440 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:47.928409 30440 tablet.cc:1620] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:47.929167 30443 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:47.929440 30443 tablet.cc:1620] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:47.930780 26513 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 01:58:47.930891 26513 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 01:58:47.932644 30447 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:47.932915 30447 tablet.cc:1620] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:47.936259 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:47.936987 29563 consensus_queue.cc:1035] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.937791 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=6cd66554f3f94d1c834ebe5b93514745] alter complete (version 2)
I20240502 01:58:47.940399 29080 raft_consensus.cc:1203] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:47.942777 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet b5439ff11ca04925bf46e736a5a53487
I20240502 01:58:47.943044 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 89a647fc0559437983278163148da132
I20240502 01:58:47.943207 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet 1c002ce2d74547399685b69a0a4cbada
I20240502 01:58:47.943526 26639 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:47.943750 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet b5439ff11ca04925bf46e736a5a53487 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:47.944205 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c002ce2d74547399685b69a0a4cbada with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:47.944617 30495 tablet_replica.cc:330] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:47.944708 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a647fc0559437983278163148da132 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:47.944761 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c002ce2d74547399685b69a0a4cbada with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:47.944873 29703 tablet_replica.cc:330] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:47.944900 30495 raft_consensus.cc:2229] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.944981 29703 raft_consensus.cc:2229] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.945076 30495 raft_consensus.cc:2258] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.945093 29703 raft_consensus.cc:2258] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.945575 29703 ts_tablet_manager.cc:1875] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.945894 30495 ts_tablet_manager.cc:1875] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.946265 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet b5439ff11ca04925bf46e736a5a53487 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:47.946583 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet b5439ff11ca04925bf46e736a5a53487 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:47.946797 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c002ce2d74547399685b69a0a4cbada with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:47.946861 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet b5439ff11ca04925bf46e736a5a53487 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:47.946925 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a647fc0559437983278163148da132 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:47.947023 29704 tablet_replica.cc:330] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:47.947060 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c002ce2d74547399685b69a0a4cbada with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:47.947082 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a647fc0559437983278163148da132 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:47 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:47.947134 29704 raft_consensus.cc:2229] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:47.947248 29702 tablet_replica.cc:330] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:47.947362 29702 raft_consensus.cc:2229] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.947379 29704 raft_consensus.cc:2258] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.947464 29702 raft_consensus.cc:2258] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.947873 29704 ts_tablet_manager.cc:1875] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.947912 29702 ts_tablet_manager.cc:1875] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.948472 30495 ts_tablet_manager.cc:1888] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.948540 30495 log.cc:1196] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/1c002ce2d74547399685b69a0a4cbada
I20240502 01:58:47.948596 29703 ts_tablet_manager.cc:1888] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:47.948659 29703 log.cc:1196] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/89a647fc0559437983278163148da132
I20240502 01:58:47.948786 30495 ts_tablet_manager.cc:1909] T 1c002ce2d74547399685b69a0a4cbada P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:47.948892 29703 ts_tablet_manager.cc:1909] T 89a647fc0559437983278163148da132 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:47.949440 29703 tablet_replica.cc:330] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:47.949563 29703 raft_consensus.cc:2229] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.949805 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 1c002ce2d74547399685b69a0a4cbada (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.949954 29702 ts_tablet_manager.cc:1888] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:47.949952 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 89a647fc0559437983278163148da132 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.950111 29704 ts_tablet_manager.cc:1888] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.950171 29704 log.cc:1196] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/b5439ff11ca04925bf46e736a5a53487
I20240502 01:58:47.950011 29702 log.cc:1196] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/89a647fc0559437983278163148da132
I20240502 01:58:47.950563 29702 ts_tablet_manager.cc:1909] T 89a647fc0559437983278163148da132 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:47.950824 29704 ts_tablet_manager.cc:1909] T b5439ff11ca04925bf46e736a5a53487 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:47.951315 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 89a647fc0559437983278163148da132 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.951401 26907 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:47.951409 29704 tablet_replica.cc:330] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:47.951452 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet b5439ff11ca04925bf46e736a5a53487 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.951524 29704 raft_consensus.cc:2229] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:47.951710 29704 raft_consensus.cc:2258] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.951682 29563 consensus_queue.cc:1035] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:47.952080 27042 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:58:47.952149 29704 ts_tablet_manager.cc:1875] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.949674 29703 raft_consensus.cc:2258] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.954375 29704 ts_tablet_manager.cc:1888] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:47.954439 29704 log.cc:1196] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/89a647fc0559437983278163148da132
I20240502 01:58:47.954680 29704 ts_tablet_manager.cc:1909] T 89a647fc0559437983278163148da132 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:47.955134 29704 tablet_replica.cc:330] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:47.955240 29704 raft_consensus.cc:2229] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.955353 29704 raft_consensus.cc:2258] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.955741 29704 ts_tablet_manager.cc:1875] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.956064 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 89a647fc0559437983278163148da132 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.957453 29704 ts_tablet_manager.cc:1888] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.957520 29704 log.cc:1196] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/1c002ce2d74547399685b69a0a4cbada
I20240502 01:58:47.957749 29704 ts_tablet_manager.cc:1909] T 1c002ce2d74547399685b69a0a4cbada P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:47.962745 29058 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:47.970103 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 1c002ce2d74547399685b69a0a4cbada (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.971241 29703 ts_tablet_manager.cc:1875] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.986498 30494 tablet_replica.cc:330] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:47.986696 30496 tablet_replica.cc:330] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:47.989114 29703 ts_tablet_manager.cc:1888] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:47.989202 29703 log.cc:1196] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/1c002ce2d74547399685b69a0a4cbada
I20240502 01:58:47.989393 30496 raft_consensus.cc:2229] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.989454 29703 ts_tablet_manager.cc:1909] T 1c002ce2d74547399685b69a0a4cbada P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:47.989604 30494 raft_consensus.cc:2229] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.989804 30496 raft_consensus.cc:2258] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.989948 30494 raft_consensus.cc:2258] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.990561 30494 ts_tablet_manager.cc:1875] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.990717 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 1c002ce2d74547399685b69a0a4cbada (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:47.990948 29703 tablet_replica.cc:330] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:47.991119 29703 raft_consensus.cc:2229] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:47.991206 30496 ts_tablet_manager.cc:1875] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.991357 29703 raft_consensus.cc:2258] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:47.991864 29703 ts_tablet_manager.cc:1875] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:47.997155 30494 ts_tablet_manager.cc:1888] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.000020 30496 ts_tablet_manager.cc:1888] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.000149 30494 log.cc:1196] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/b5439ff11ca04925bf46e736a5a53487
I20240502 01:58:48.000466 30494 ts_tablet_manager.cc:1909] T b5439ff11ca04925bf46e736a5a53487 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:48.000568 30496 log.cc:1196] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/b5439ff11ca04925bf46e736a5a53487
I20240502 01:58:48.001596 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet b5439ff11ca04925bf46e736a5a53487 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:48.001960 30496 ts_tablet_manager.cc:1909] T b5439ff11ca04925bf46e736a5a53487 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.002588 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:48.003203 30496 tablet_replica.cc:330] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:48.003288 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet b5439ff11ca04925bf46e736a5a53487 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:48.003578 30496 raft_consensus.cc:2229] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.003903 30496 raft_consensus.cc:2258] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.004432 30496 ts_tablet_manager.cc:1875] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.005820 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:48.012071 26907 tablet_service.cc:1467] Processing CreateTablet for tablet d14b1389d460401b83d3652b25aaddec (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.012341 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14b1389d460401b83d3652b25aaddec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.012744 29703 ts_tablet_manager.cc:1888] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.012804 29703 log.cc:1196] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/b5439ff11ca04925bf46e736a5a53487
I20240502 01:58:48.013027 29703 ts_tablet_manager.cc:1909] T b5439ff11ca04925bf46e736a5a53487 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.014139 30501 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:48.014726 29330 tablet_service.cc:1467] Processing CreateTablet for tablet d14b1389d460401b83d3652b25aaddec (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.014806 29058 tablet_service.cc:1467] Processing CreateTablet for tablet b1de60ca7a73404abcc530b17bc20662 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.014981 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14b1389d460401b83d3652b25aaddec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.015031 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1de60ca7a73404abcc530b17bc20662. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.016254 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet b5439ff11ca04925bf46e736a5a53487 (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:48.016786 26639 tablet_service.cc:1467] Processing CreateTablet for tablet 1c851506649546cb96d13a8c4ea33797 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.017038 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c851506649546cb96d13a8c4ea33797. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.019122 30501 tablet_bootstrap.cc:654] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.020037 30501 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:48.020040 29454 tablet_service.cc:1467] Processing CreateTablet for tablet b1de60ca7a73404abcc530b17bc20662 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.020123 30501 ts_tablet_manager.cc:1367] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:48.020272 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1de60ca7a73404abcc530b17bc20662. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.020323 27042 tablet_service.cc:1467] Processing CreateTablet for tablet b1de60ca7a73404abcc530b17bc20662 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.020540 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1de60ca7a73404abcc530b17bc20662. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.020503 30501 raft_consensus.cc:348] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.020599 30501 raft_consensus.cc:374] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.020627 30501 raft_consensus.cc:729] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.020735 30501 consensus_queue.cc:260] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.020979 30501 ts_tablet_manager.cc:1398] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:48.022162 30506 tablet_bootstrap.cc:492] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:48.022272 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 1c851506649546cb96d13a8c4ea33797 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.022505 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c851506649546cb96d13a8c4ea33797. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.022450 30507 tablet_bootstrap.cc:492] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:48.023108 30506 tablet_bootstrap.cc:654] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.023447 30507 tablet_bootstrap.cc:654] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.023871 29195 tablet_service.cc:1467] Processing CreateTablet for tablet d14b1389d460401b83d3652b25aaddec (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.024085 30506 tablet_bootstrap.cc:492] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:48.024118 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14b1389d460401b83d3652b25aaddec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.024156 30506 ts_tablet_manager.cc:1367] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.024541 30506 raft_consensus.cc:348] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.024642 30506 raft_consensus.cc:374] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.024677 30506 raft_consensus.cc:729] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.024789 30506 consensus_queue.cc:260] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.025043 30506 ts_tablet_manager.cc:1398] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.025202 30506 tablet_bootstrap.cc:492] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:48.025983 30506 tablet_bootstrap.cc:654] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.026270 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 1c851506649546cb96d13a8c4ea33797 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.026479 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c851506649546cb96d13a8c4ea33797. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.028748 30496 ts_tablet_manager.cc:1888] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.028867 30496 log.cc:1196] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/1c002ce2d74547399685b69a0a4cbada
I20240502 01:58:48.029136 30496 ts_tablet_manager.cc:1909] T 1c002ce2d74547399685b69a0a4cbada P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.029773 30503 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:48.030128 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 1c002ce2d74547399685b69a0a4cbada (table rebalance_test_table_6 [id=d341bb6a432647ab90b4207f29bfa6f2]) successfully deleted
I20240502 01:58:48.030761 30503 tablet_bootstrap.cc:654] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.031384 30502 tablet_bootstrap.cc:492] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:48.032235 30502 tablet_bootstrap.cc:654] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.032984 30503 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:48.033066 30503 ts_tablet_manager.cc:1367] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:48.033458 30503 raft_consensus.cc:348] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.033951 30502 tablet_bootstrap.cc:492] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:48.034086 30502 ts_tablet_manager.cc:1367] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:48.034472 30502 raft_consensus.cc:348] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.034577 30502 raft_consensus.cc:374] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.034602 30502 raft_consensus.cc:729] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.034683 30503 raft_consensus.cc:374] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.034763 30503 raft_consensus.cc:729] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.034910 30503 consensus_queue.cc:260] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.036612 30503 ts_tablet_manager.cc:1398] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:48.036940 30502 consensus_queue.cc:260] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.037817 30502 ts_tablet_manager.cc:1398] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:48.038326 30504 tablet_bootstrap.cc:492] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:48.039100 30504 tablet_bootstrap.cc:654] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.039397 30511 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:48.040194 30511 tablet_bootstrap.cc:654] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.042424 30506 tablet_bootstrap.cc:492] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:48.042505 30506 ts_tablet_manager.cc:1367] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:48.042927 30506 raft_consensus.cc:348] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.043015 30506 raft_consensus.cc:374] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.043040 30506 raft_consensus.cc:729] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.043141 30506 consensus_queue.cc:260] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.043390 30506 ts_tablet_manager.cc:1398] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.043682 30511 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:48.043773 30511 ts_tablet_manager.cc:1367] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:48.045140 30511 raft_consensus.cc:348] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.045248 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:48.049343 30511 raft_consensus.cc:374] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.049465 30511 raft_consensus.cc:729] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.049670 30511 consensus_queue.cc:260] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.050441 30511 ts_tablet_manager.cc:1398] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:48.050642 30511 tablet_bootstrap.cc:492] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:48.050940 30440 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:48.051517 30511 tablet_bootstrap.cc:654] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.052582 30511 tablet_bootstrap.cc:492] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:48.052712 30511 ts_tablet_manager.cc:1367] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:48.053063 30443 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:48.053859 30447 tablet.cc:1722] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:48.054296 29349 raft_consensus.cc:1203] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.054956 30504 tablet_bootstrap.cc:492] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:48.055099 30504 ts_tablet_manager.cc:1367] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:48.055553 30504 raft_consensus.cc:348] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.055728 30504 raft_consensus.cc:374] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.055809 30504 raft_consensus.cc:729] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.055971 30504 consensus_queue.cc:260] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.057992 30504 ts_tablet_manager.cc:1398] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:48.058321 30511 raft_consensus.cc:348] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.058480 30511 raft_consensus.cc:374] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.058553 30511 raft_consensus.cc:729] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.058737 30511 consensus_queue.cc:260] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.059118 30511 ts_tablet_manager.cc:1398] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:48.060813 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=6cd66554f3f94d1c834ebe5b93514745] alter complete (version 3)
I20240502 01:58:48.063143 30507 tablet_bootstrap.cc:492] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:48.063249 30507 ts_tablet_manager.cc:1367] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240502 01:58:48.063614 30507 raft_consensus.cc:348] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.063705 30507 raft_consensus.cc:374] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.063735 30507 raft_consensus.cc:729] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.063833 30507 consensus_queue.cc:260] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.064070 30507 ts_tablet_manager.cc:1398] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.120986 29489 raft_consensus.cc:1203] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.145151 29216 raft_consensus.cc:1203] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.162590 26510 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:48.162703 26510 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:48.164561 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 708088176cfa412a8246fd55f2fb9088
I20240502 01:58:48.165122 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 708088176cfa412a8246fd55f2fb9088 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:48.165155 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 708088176cfa412a8246fd55f2fb9088 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:48.165268 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet 708088176cfa412a8246fd55f2fb9088 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:48.165455 30560 tablet_replica.cc:330] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:48.165521 30561 tablet_replica.cc:330] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:48.165578 30560 raft_consensus.cc:2229] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.165622 30494 tablet_replica.cc:330] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:48.165625 30561 raft_consensus.cc:2229] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.165692 30560 raft_consensus.cc:2258] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.165757 30494 raft_consensus.cc:2229] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.165810 30561 raft_consensus.cc:2258] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.165864 30494 raft_consensus.cc:2258] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.166424 30560 ts_tablet_manager.cc:1875] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.166424 30561 ts_tablet_manager.cc:1875] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.166457 30494 ts_tablet_manager.cc:1875] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.168437 30494 ts_tablet_manager.cc:1888] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.168486 30494 log.cc:1196] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/708088176cfa412a8246fd55f2fb9088
I20240502 01:58:48.168529 30561 ts_tablet_manager.cc:1888] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.168558 30560 ts_tablet_manager.cc:1888] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.168596 30561 log.cc:1196] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/708088176cfa412a8246fd55f2fb9088
I20240502 01:58:48.168612 30560 log.cc:1196] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/708088176cfa412a8246fd55f2fb9088
I20240502 01:58:48.168651 30494 ts_tablet_manager.cc:1909] T 708088176cfa412a8246fd55f2fb9088 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:48.168840 30561 ts_tablet_manager.cc:1909] T 708088176cfa412a8246fd55f2fb9088 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:48.168923 30560 ts_tablet_manager.cc:1909] T 708088176cfa412a8246fd55f2fb9088 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:48.169413 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table_new_name [id=6cd66554f3f94d1c834ebe5b93514745]) successfully deleted
I20240502 01:58:48.169495 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table_new_name [id=6cd66554f3f94d1c834ebe5b93514745]) successfully deleted
I20240502 01:58:48.169626 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 708088176cfa412a8246fd55f2fb9088 (table rebalancer_dynamic_table_new_name [id=6cd66554f3f94d1c834ebe5b93514745]) successfully deleted
I20240502 01:58:48.181872 29080 raft_consensus.cc:1203] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.196753 30500 raft_consensus.cc:482] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:48.196851 30500 raft_consensus.cc:504] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.197245 30500 leader_election.cc:288] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:48.197541 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14b1389d460401b83d3652b25aaddec" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:48.197681 29349 raft_consensus.cc:2454] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:48.197975 26859 leader_election.cc:302] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:48.198348 30499 raft_consensus.cc:2781] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:48.198415 30499 raft_consensus.cc:482] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:48.198482 30499 raft_consensus.cc:3037] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.199455 30499 raft_consensus.cc:504] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.199764 30499 leader_election.cc:288] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:48.200121 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14b1389d460401b83d3652b25aaddec" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:48.200222 29349 raft_consensus.cc:3037] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.201076 29349 raft_consensus.cc:2454] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:48.201117 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14b1389d460401b83d3652b25aaddec" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:48.201182 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14b1389d460401b83d3652b25aaddec" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:48.201252 29216 raft_consensus.cc:2454] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:48.201727 26859 leader_election.cc:302] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:48.201779 29349 raft_consensus.cc:1203] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.201918 30500 raft_consensus.cc:2781] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:48.201992 30500 raft_consensus.cc:686] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:48.202121 30500 consensus_queue.cc:237] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.203843 26510 catalog_manager.cc:5484] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:48.251305 29216 raft_consensus.cc:1203] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.254984 29489 raft_consensus.cc:1203] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.257752 29080 raft_consensus.cc:1203] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.266183 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:48.270920 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 59502898034c4f6aa3a2fb98ccd225e1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691]), partition=
I20240502 01:58:48.271057 27042 tablet_service.cc:1467] Processing CreateTablet for tablet 59502898034c4f6aa3a2fb98ccd225e1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691]), partition=
I20240502 01:58:48.271178 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59502898034c4f6aa3a2fb98ccd225e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.271302 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59502898034c4f6aa3a2fb98ccd225e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.271386 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 59502898034c4f6aa3a2fb98ccd225e1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691]), partition=
I20240502 01:58:48.271591 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59502898034c4f6aa3a2fb98ccd225e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.273278 30503 tablet_bootstrap.cc:492] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:48.273371 30507 tablet_bootstrap.cc:492] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:48.273989 30564 tablet_bootstrap.cc:492] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:48.274212 30507 tablet_bootstrap.cc:654] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.274240 29349 raft_consensus.cc:1203] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.274499 30503 tablet_bootstrap.cc:654] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.274762 30564 tablet_bootstrap.cc:654] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.275514 30503 tablet_bootstrap.cc:492] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:48.275602 30503 ts_tablet_manager.cc:1367] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:48.276067 30503 raft_consensus.cc:348] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.276182 30503 raft_consensus.cc:374] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.276216 30503 raft_consensus.cc:729] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.276327 30503 consensus_queue.cc:260] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.276595 30564 tablet_bootstrap.cc:492] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:48.276621 30503 ts_tablet_manager.cc:1398] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.276708 30564 ts_tablet_manager.cc:1367] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:48.277071 30507 tablet_bootstrap.cc:492] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:48.277166 30507 ts_tablet_manager.cc:1367] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:48.277104 30564 raft_consensus.cc:348] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.277597 30507 raft_consensus.cc:348] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.277724 30507 raft_consensus.cc:374] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.277766 30507 raft_consensus.cc:729] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.277902 30507 consensus_queue.cc:260] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.277261 30564 raft_consensus.cc:374] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.278000 30564 raft_consensus.cc:729] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.278216 30507 ts_tablet_manager.cc:1398] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.278172 30564 consensus_queue.cc:260] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.278280 29564 raft_consensus.cc:482] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:48.278365 29564 raft_consensus.cc:504] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.278447 30564 ts_tablet_manager.cc:1398] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.278712 29564 leader_election.cc:288] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:48.279093 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59502898034c4f6aa3a2fb98ccd225e1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:48.279158 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59502898034c4f6aa3a2fb98ccd225e1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:48.279194 29349 raft_consensus.cc:2454] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:48.279258 27063 raft_consensus.cc:2454] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:48.279835 26727 leader_election.cc:302] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:48.280051 29826 raft_consensus.cc:2781] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:48.280105 29826 raft_consensus.cc:482] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:48.280141 29826 raft_consensus.cc:3037] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.280998 29826 raft_consensus.cc:504] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.281289 29826 leader_election.cc:288] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:48.281584 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59502898034c4f6aa3a2fb98ccd225e1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:48.281683 29349 raft_consensus.cc:3037] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.281832 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59502898034c4f6aa3a2fb98ccd225e1" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:48.281916 27063 raft_consensus.cc:3037] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.282542 29349 raft_consensus.cc:2454] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:48.282769 27063 raft_consensus.cc:2454] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:48.282812 26727 leader_election.cc:302] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:48.282959 29826 raft_consensus.cc:2781] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:48.283046 29826 raft_consensus.cc:686] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:48.283159 29826 consensus_queue.cc:237] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:48.288125 26511 catalog_manager.cc:5484] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:48.317262 29726 raft_consensus.cc:482] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:48.317361 29726 raft_consensus.cc:504] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.317761 29726 leader_election.cc:288] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:48.318097 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c851506649546cb96d13a8c4ea33797" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:48.318233 26661 raft_consensus.cc:2454] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:48.318533 29149 leader_election.cc:302] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:48.318712 29726 raft_consensus.cc:2781] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:48.318780 29726 raft_consensus.cc:482] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:48.318838 29726 raft_consensus.cc:3037] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.319845 29726 raft_consensus.cc:504] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.320219 29726 leader_election.cc:288] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:48.320495 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c851506649546cb96d13a8c4ea33797" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:48.320621 26661 raft_consensus.cc:3037] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.321544 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c851506649546cb96d13a8c4ea33797" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:48.321578 26661 raft_consensus.cc:2454] T 1c851506649546cb96d13a8c4ea33797 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:48.321626 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c851506649546cb96d13a8c4ea33797" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:48.321694 29489 raft_consensus.cc:2454] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:48.321859 29149 leader_election.cc:302] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:48.322010 29726 raft_consensus.cc:2781] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:48.322165 29726 raft_consensus.cc:686] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:48.322297 29726 consensus_queue.cc:237] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.322791 29213 raft_consensus.cc:1203] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:48.323827 26507 catalog_manager.cc:5484] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:48.339740 29997 raft_consensus.cc:482] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:48.339838 29997 raft_consensus.cc:504] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.340196 29997 leader_election.cc:288] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:48.340850 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1de60ca7a73404abcc530b17bc20662" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:48.340972 27063 raft_consensus.cc:2454] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:48.341217 29016 leader_election.cc:302] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:48.341400 29997 raft_consensus.cc:2781] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:48.341461 29997 raft_consensus.cc:482] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:48.341500 29997 raft_consensus.cc:3037] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.342424 29997 raft_consensus.cc:504] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.342727 29997 leader_election.cc:288] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:48.343125 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1de60ca7a73404abcc530b17bc20662" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:48.343245 27063 raft_consensus.cc:3037] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.343964 27063 raft_consensus.cc:2454] T b1de60ca7a73404abcc530b17bc20662 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:48.344246 29016 leader_election.cc:302] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:48.344425 29997 raft_consensus.cc:2781] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:48.344409 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1de60ca7a73404abcc530b17bc20662" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:48.344475 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1de60ca7a73404abcc530b17bc20662" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:48.344508 29997 raft_consensus.cc:686] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:48.344529 29488 raft_consensus.cc:2454] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:48.344627 29997 consensus_queue.cc:237] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:48.346096 26507 catalog_manager.cc:5484] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:48.356544 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:48.361425 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:48.364394 27042 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:58:48.364398 26907 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:48.368327 29058 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:48.386830 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:48.391775 26639 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:48.392493 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:48.394476 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:48.401176 29349 raft_consensus.cc:1261] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:48.402134 27063 raft_consensus.cc:1261] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:48.402618 29826 consensus_queue.cc:1035] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.403352 29826 consensus_queue.cc:1035] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.406131 26507 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 59502898034c4f6aa3a2fb98ccd225e1 (table rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691]). Expected version 1 got 0
I20240502 01:58:48.406515 26511 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 59502898034c4f6aa3a2fb98ccd225e1 (table rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691]). Expected version 1 got 0
I20240502 01:58:48.408769 30587 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:48.409004 26507 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 59502898034c4f6aa3a2fb98ccd225e1 (table rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691]). Expected version 1 got 0
I20240502 01:58:48.409724 30587 tablet.cc:1620] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:48.415052 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691] alter complete (version 1)
I20240502 01:58:48.421658 30588 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:48.422453 30588 tablet.cc:1620] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:58:48.422724 30440 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:48.423002 30440 tablet.cc:1620] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:48.444491 26511 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 01:58:48.444597 26511 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 01:58:48.446638 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 38c27ffed0ec4d2ca8525682b609369e
I20240502 01:58:48.446923 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 55fcea76c3c24decb635ffd912d0a738
I20240502 01:58:48.447074 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c27ffed0ec4d2ca8525682b609369e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.447181 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet cc7e7947fe8a466da4778d274184ac9c
I20240502 01:58:48.447263 29703 tablet_replica.cc:330] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.447414 29703 raft_consensus.cc:2229] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.447549 29703 raft_consensus.cc:2258] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.447551 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 55fcea76c3c24decb635ffd912d0a738 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:48.447582 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c27ffed0ec4d2ca8525682b609369e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:48.447719 30496 tablet_replica.cc:330] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:48.447825 30496 raft_consensus.cc:2229] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.447983 30496 raft_consensus.cc:2258] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.448041 29703 ts_tablet_manager.cc:1875] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.448197 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet cc7e7947fe8a466da4778d274184ac9c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.448446 30496 ts_tablet_manager.cc:1875] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.449831 29703 ts_tablet_manager.cc:1888] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.449891 29703 log.cc:1196] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/38c27ffed0ec4d2ca8525682b609369e
I20240502 01:58:48.450130 29703 ts_tablet_manager.cc:1909] T 38c27ffed0ec4d2ca8525682b609369e P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.450551 30496 ts_tablet_manager.cc:1888] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.450608 30496 log.cc:1196] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/55fcea76c3c24decb635ffd912d0a738
I20240502 01:58:48.450646 29703 tablet_replica.cc:330] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.450760 29703 raft_consensus.cc:2229] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.450839 30496 ts_tablet_manager.cc:1909] T 55fcea76c3c24decb635ffd912d0a738 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.450881 29703 raft_consensus.cc:2258] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.451164 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 38c27ffed0ec4d2ca8525682b609369e (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.451270 29703 ts_tablet_manager.cc:1875] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.451345 30496 tablet_replica.cc:330] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:48.451454 30496 raft_consensus.cc:2229] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.451582 30496 raft_consensus.cc:2258] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.451614 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 55fcea76c3c24decb635ffd912d0a738 (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.451968 30496 ts_tablet_manager.cc:1875] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.452739 29703 ts_tablet_manager.cc:1888] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.452783 29703 log.cc:1196] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/cc7e7947fe8a466da4778d274184ac9c
I20240502 01:58:48.452936 29703 ts_tablet_manager.cc:1909] T cc7e7947fe8a466da4778d274184ac9c P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.453302 30496 ts_tablet_manager.cc:1888] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.453362 30496 log.cc:1196] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/38c27ffed0ec4d2ca8525682b609369e
I20240502 01:58:48.453410 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet cc7e7947fe8a466da4778d274184ac9c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:48.453523 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 55fcea76c3c24decb635ffd912d0a738 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.453624 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c27ffed0ec4d2ca8525682b609369e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:48.453758 29704 tablet_replica.cc:330] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:48.453825 30496 ts_tablet_manager.cc:1909] T 38c27ffed0ec4d2ca8525682b609369e P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.453860 29704 raft_consensus.cc:2229] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.453945 29704 raft_consensus.cc:2258] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.453931 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet cc7e7947fe8a466da4778d274184ac9c (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.454356 29704 ts_tablet_manager.cc:1875] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.454427 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 38c27ffed0ec4d2ca8525682b609369e (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.456137 29704 ts_tablet_manager.cc:1888] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.456192 29704 log.cc:1196] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/cc7e7947fe8a466da4778d274184ac9c
I20240502 01:58:48.456429 29704 ts_tablet_manager.cc:1909] T cc7e7947fe8a466da4778d274184ac9c P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:48.456923 29704 tablet_replica.cc:330] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:48.457036 29704 raft_consensus.cc:2229] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.457134 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet cc7e7947fe8a466da4778d274184ac9c (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.457208 29704 raft_consensus.cc:2258] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.457311 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet cc7e7947fe8a466da4778d274184ac9c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.457581 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet 55fcea76c3c24decb635ffd912d0a738 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.457641 29704 ts_tablet_manager.cc:1875] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.457779 30598 tablet_replica.cc:330] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.457870 29703 tablet_replica.cc:330] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.457916 30598 raft_consensus.cc:2229] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.458068 30598 raft_consensus.cc:2258] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.458083 29703 raft_consensus.cc:2229] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.458323 29703 raft_consensus.cc:2258] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.458545 30598 ts_tablet_manager.cc:1875] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.459452 29703 ts_tablet_manager.cc:1875] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.459514 29704 ts_tablet_manager.cc:1888] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.459640 29704 log.cc:1196] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/38c27ffed0ec4d2ca8525682b609369e
I20240502 01:58:48.459942 29704 ts_tablet_manager.cc:1909] T 38c27ffed0ec4d2ca8525682b609369e P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:48.460562 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 38c27ffed0ec4d2ca8525682b609369e (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.460690 30598 ts_tablet_manager.cc:1888] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.460845 30598 log.cc:1196] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/55fcea76c3c24decb635ffd912d0a738
I20240502 01:58:48.461077 30598 ts_tablet_manager.cc:1909] T 55fcea76c3c24decb635ffd912d0a738 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:48.461581 30598 tablet_replica.cc:330] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.461695 30598 raft_consensus.cc:2229] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.461819 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 55fcea76c3c24decb635ffd912d0a738 (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.461855 30598 raft_consensus.cc:2258] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.461870 29703 ts_tablet_manager.cc:1888] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.461978 29703 log.cc:1196] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/55fcea76c3c24decb635ffd912d0a738
I20240502 01:58:48.462236 29703 ts_tablet_manager.cc:1909] T 55fcea76c3c24decb635ffd912d0a738 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.462354 30598 ts_tablet_manager.cc:1875] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.462929 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 55fcea76c3c24decb635ffd912d0a738 (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.464023 30598 ts_tablet_manager.cc:1888] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.464097 30598 log.cc:1196] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/cc7e7947fe8a466da4778d274184ac9c
I20240502 01:58:48.464314 30598 ts_tablet_manager.cc:1909] T cc7e7947fe8a466da4778d274184ac9c P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:48.464802 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet cc7e7947fe8a466da4778d274184ac9c (table rebalance_test_table_8 [id=fe6318fa9234475b94b8597d4f9fe9d1]) successfully deleted
I20240502 01:58:48.465996 26794 consensus_queue.cc:237] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:48.467216 26927 raft_consensus.cc:1261] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:48.467226 29488 raft_consensus.cc:1261] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:48.467605 29826 consensus_queue.cc:1035] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:48.467782 29564 consensus_queue.cc:1035] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:48.469288 29826 raft_consensus.cc:2932] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:48.469719 26927 raft_consensus.cc:2932] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:48.470675 29488 raft_consensus.cc:2932] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:48.473290 26511 catalog_manager.cc:5484] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:48.475991 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.476227 29703 tablet_replica.cc:330] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.476343 29703 raft_consensus.cc:2229] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.476460 29703 raft_consensus.cc:2258] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.476903 29703 ts_tablet_manager.cc:1875] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:48.478641 29703 ts_tablet_manager.cc:1888] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:48.478713 29703 log.cc:1196] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/6e2ebe00a52240cb9ac5fc8fea9d7151
I20240502 01:58:48.479118 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:48.523479 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:48.529445 30587 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:48.529739 30587 tablet.cc:1620] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:48.530073 30588 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:48.530264 30440 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:48.530356 30588 tablet.cc:1620] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:58:48.530524 30440 tablet.cc:1620] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:48.535578 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=f9b09c47afb0405aa4660d7da7163691] alter complete (version 2)
I20240502 01:58:48.554224 27042 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:58:48.566921 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:48.568585 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:48.570381 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:48.578521 29058 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:48.579373 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:48.585929 26639 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:48.591727 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:48.638978 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:48.645023 30587 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:48.645334 30588 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:48.645488 30440 tablet.cc:1722] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:48.649634 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=f9b09c47afb0405aa4660d7da7163691] alter complete (version 3)
I20240502 01:58:48.652393 27064 consensus_queue.cc:237] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.653424 26661 raft_consensus.cc:1261] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:48.653612 29080 raft_consensus.cc:1261] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:48.653771 28915 consensus_queue.cc:1035] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:48.653981 29625 consensus_queue.cc:1035] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:48.655705 30680 raft_consensus.cc:2932] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:48.656008 26661 raft_consensus.cc:2932] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:48.656332 29080 raft_consensus.cc:2932] T 936e77ced55c41d6bfe955d9a4271b80 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:48.659117 26513 catalog_manager.cc:5484] T 936e77ced55c41d6bfe955d9a4271b80 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:48.662544 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 936e77ced55c41d6bfe955d9a4271b80 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.662978 30598 tablet_replica.cc:330] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.663125 30598 raft_consensus.cc:2229] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.663318 30598 raft_consensus.cc:2258] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.663789 30598 ts_tablet_manager.cc:1875] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:48.665493 30598 ts_tablet_manager.cc:1888] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:48.665591 30598 log.cc:1196] T 936e77ced55c41d6bfe955d9a4271b80 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/936e77ced55c41d6bfe955d9a4271b80
I20240502 01:58:48.666050 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 936e77ced55c41d6bfe955d9a4271b80 (table rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]) successfully deleted
I20240502 01:58:48.666613 29213 raft_consensus.cc:3037] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.667814 30500 consensus_queue.cc:1035] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.670476 30500 consensus_queue.cc:1035] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.704308 29726 consensus_queue.cc:1035] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.707705 29488 raft_consensus.cc:3037] T 1c851506649546cb96d13a8c4ea33797 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.708958 30569 consensus_queue.cc:1035] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.737147 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:48.740339 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:48.741075 27042 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:58:48.750485 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:48.751046 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:48.752804 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:48.752933 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:48.753103 26639 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:48.755808 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 59502898034c4f6aa3a2fb98ccd225e1
I20240502 01:58:48.755849 29058 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:48.756269 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 59502898034c4f6aa3a2fb98ccd225e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:48.756932 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 59502898034c4f6aa3a2fb98ccd225e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:48.757302 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 59502898034c4f6aa3a2fb98ccd225e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:48.757382 29704 tablet_replica.cc:330] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:48.757460 30496 tablet_replica.cc:330] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:48.757511 29704 raft_consensus.cc:2229] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.757581 30496 raft_consensus.cc:2229] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.757615 29704 raft_consensus.cc:2258] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.757753 30496 raft_consensus.cc:2258] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.758464 30748 tablet_replica.cc:330] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:48.759186 30496 ts_tablet_manager.cc:1875] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.761029 30496 ts_tablet_manager.cc:1888] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.761082 30496 log.cc:1196] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/59502898034c4f6aa3a2fb98ccd225e1
I20240502 01:58:48.761255 30496 ts_tablet_manager.cc:1909] T 59502898034c4f6aa3a2fb98ccd225e1 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.761904 29704 ts_tablet_manager.cc:1875] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.762133 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 59502898034c4f6aa3a2fb98ccd225e1 (table rebalancer_dynamic_table_new_name [id=f9b09c47afb0405aa4660d7da7163691]) successfully deleted
I20240502 01:58:48.764042 29704 ts_tablet_manager.cc:1888] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.764111 29704 log.cc:1196] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/59502898034c4f6aa3a2fb98ccd225e1
I20240502 01:58:48.764521 29704 ts_tablet_manager.cc:1909] T 59502898034c4f6aa3a2fb98ccd225e1 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:48.765210 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 59502898034c4f6aa3a2fb98ccd225e1 (table rebalancer_dynamic_table_new_name [id=f9b09c47afb0405aa4660d7da7163691]) successfully deleted
I20240502 01:58:48.768474 30748 raft_consensus.cc:2229] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.768651 30748 raft_consensus.cc:2258] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.769218 30748 ts_tablet_manager.cc:1875] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.771109 30748 ts_tablet_manager.cc:1888] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:48.771246 30748 log.cc:1196] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/59502898034c4f6aa3a2fb98ccd225e1
I20240502 01:58:48.771523 30748 ts_tablet_manager.cc:1909] T 59502898034c4f6aa3a2fb98ccd225e1 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:48.772225 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 59502898034c4f6aa3a2fb98ccd225e1 (table rebalancer_dynamic_table_new_name [id=f9b09c47afb0405aa4660d7da7163691]) successfully deleted
I20240502 01:58:48.794567 26775 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:58:48.805339 29488 raft_consensus.cc:3037] T b1de60ca7a73404abcc530b17bc20662 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.806843 29563 consensus_queue.cc:1035] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.815824 29563 consensus_queue.cc:1035] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:48.836535 27063 consensus_queue.cc:237] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:48.837690 26661 raft_consensus.cc:1261] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:48.837690 29213 raft_consensus.cc:1261] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:48.838068 30680 consensus_queue.cc:1035] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:48.838253 29625 consensus_queue.cc:1035] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:48.839833 30680 raft_consensus.cc:2932] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:48.840543 29213 raft_consensus.cc:2932] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:48.842715 26514 catalog_manager.cc:5484] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:48.844257 26661 raft_consensus.cc:2932] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:48.847975 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet e60816eacfb547cda5a6c87c68bf101d with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.848212 30598 tablet_replica.cc:330] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.848361 30598 raft_consensus.cc:2229] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.848500 30598 raft_consensus.cc:2258] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.848943 30598 ts_tablet_manager.cc:1875] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:48.851024 30598 ts_tablet_manager.cc:1888] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:48.851097 30598 log.cc:1196] T e60816eacfb547cda5a6c87c68bf101d P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/e60816eacfb547cda5a6c87c68bf101d
I20240502 01:58:48.851495 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet e60816eacfb547cda5a6c87c68bf101d (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:48.858224 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:48.862846 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 751e68570b084b8382751ee3e1cfb727 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270]), partition=
I20240502 01:58:48.862882 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 751e68570b084b8382751ee3e1cfb727 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270]), partition=
I20240502 01:58:48.863083 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751e68570b084b8382751ee3e1cfb727. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.863101 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751e68570b084b8382751ee3e1cfb727. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.864925 30783 tablet_bootstrap.cc:492] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:48.865363 30784 tablet_bootstrap.cc:492] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:48.865491 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 751e68570b084b8382751ee3e1cfb727 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270]), partition=
I20240502 01:58:48.865744 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751e68570b084b8382751ee3e1cfb727. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.866206 30784 tablet_bootstrap.cc:654] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.867177 30784 tablet_bootstrap.cc:492] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:48.867249 30784 ts_tablet_manager.cc:1367] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.867636 30784 raft_consensus.cc:348] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.867739 30784 raft_consensus.cc:374] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.867769 30784 raft_consensus.cc:729] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.867889 30784 consensus_queue.cc:260] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.868726 30784 ts_tablet_manager.cc:1398] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.869462 30783 tablet_bootstrap.cc:654] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.870213 30786 tablet_bootstrap.cc:492] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:48.871044 30786 tablet_bootstrap.cc:654] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.872006 30786 tablet_bootstrap.cc:492] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:48.872143 30786 ts_tablet_manager.cc:1367] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:48.872300 30783 tablet_bootstrap.cc:492] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:48.872433 30783 ts_tablet_manager.cc:1367] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:58:48.872704 30786 raft_consensus.cc:348] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.872879 30786 raft_consensus.cc:374] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.872961 30786 raft_consensus.cc:729] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.872932 30783 raft_consensus.cc:348] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.873028 30783 raft_consensus.cc:374] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.873119 30783 raft_consensus.cc:729] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.873121 30786 consensus_queue.cc:260] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.873299 30783 consensus_queue.cc:260] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.873607 30786 ts_tablet_manager.cc:1398] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.873628 30783 ts_tablet_manager.cc:1398] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.900151 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:48.911486 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 8afd4c5362464c4cba43d5ef28b50c29 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.911765 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8afd4c5362464c4cba43d5ef28b50c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.911778 29196 tablet_service.cc:1467] Processing CreateTablet for tablet c2563da5db7a4e5197532b31e9aef636 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.912024 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2563da5db7a4e5197532b31e9aef636. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.912478 26639 tablet_service.cc:1467] Processing CreateTablet for tablet a326a7899c064c95b426aae61d65208f (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.912750 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a326a7899c064c95b426aae61d65208f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.913794 30783 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:48.914319 29454 tablet_service.cc:1467] Processing CreateTablet for tablet a326a7899c064c95b426aae61d65208f (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.914301 27042 tablet_service.cc:1467] Processing CreateTablet for tablet 8afd4c5362464c4cba43d5ef28b50c29 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.914521 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a326a7899c064c95b426aae61d65208f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.914552 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8afd4c5362464c4cba43d5ef28b50c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.914602 30783 tablet_bootstrap.cc:654] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.917243 30821 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:48.918061 29060 tablet_service.cc:1467] Processing CreateTablet for tablet 8afd4c5362464c4cba43d5ef28b50c29 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.911486 29451 tablet_service.cc:1467] Processing CreateTablet for tablet c2563da5db7a4e5197532b31e9aef636 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.918249 30821 tablet_bootstrap.cc:654] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.918311 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2563da5db7a4e5197532b31e9aef636. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.918323 29060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8afd4c5362464c4cba43d5ef28b50c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.919575 30821 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:48.919646 30821 ts_tablet_manager.cc:1367] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.920109 30821 raft_consensus.cc:348] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.920200 30821 raft_consensus.cc:374] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.920229 30821 raft_consensus.cc:729] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.920323 30784 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:48.918071 29059 tablet_service.cc:1467] Processing CreateTablet for tablet a326a7899c064c95b426aae61d65208f (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.918071 29058 tablet_service.cc:1467] Processing CreateTablet for tablet c2563da5db7a4e5197532b31e9aef636 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:48.920598 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a326a7899c064c95b426aae61d65208f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.920563 30821 consensus_queue.cc:260] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.920919 30821 ts_tablet_manager.cc:1398] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.921075 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2563da5db7a4e5197532b31e9aef636. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:48.921149 30784 tablet_bootstrap.cc:654] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.922142 30784 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:48.922209 30784 ts_tablet_manager.cc:1367] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.922617 30784 raft_consensus.cc:348] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.922704 30784 raft_consensus.cc:374] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.922729 30784 raft_consensus.cc:729] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.922827 30784 consensus_queue.cc:260] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.923074 30784 ts_tablet_manager.cc:1398] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.923221 30784 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:48.923964 30784 tablet_bootstrap.cc:654] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.924899 30784 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:48.925042 30784 ts_tablet_manager.cc:1367] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:48.925434 30784 raft_consensus.cc:348] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:48.925527 30784 raft_consensus.cc:374] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.925559 30784 raft_consensus.cc:729] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.925665 30784 consensus_queue.cc:260] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:48.926272 30784 ts_tablet_manager.cc:1398] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.927007 30784 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:48.927021 30818 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:48.928086 30818 tablet_bootstrap.cc:654] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.928086 30784 tablet_bootstrap.cc:654] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.929615 30818 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:48.929738 30818 ts_tablet_manager.cc:1367] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:48.930198 30819 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:48.930155 30818 raft_consensus.cc:348] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.930251 30818 raft_consensus.cc:374] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.930331 30818 raft_consensus.cc:729] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.930518 30818 consensus_queue.cc:260] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.930636 30783 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:48.930723 30783 ts_tablet_manager.cc:1367] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:48.931075 30783 raft_consensus.cc:348] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.931155 30819 tablet_bootstrap.cc:654] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.931166 30783 raft_consensus.cc:374] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.931196 30783 raft_consensus.cc:729] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.931301 30783 consensus_queue.cc:260] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.931496 30818 ts_tablet_manager.cc:1398] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.931545 30783 ts_tablet_manager.cc:1398] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.931700 30783 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:48.932467 30783 tablet_bootstrap.cc:654] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.932602 30784 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:48.932672 30784 ts_tablet_manager.cc:1367] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:48.933049 30784 raft_consensus.cc:348] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.933140 30784 raft_consensus.cc:374] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.933182 30784 raft_consensus.cc:729] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.933295 30784 consensus_queue.cc:260] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.933533 30784 ts_tablet_manager.cc:1398] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:48.933882 29563 raft_consensus.cc:482] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:48.933951 29563 raft_consensus.cc:504] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.934244 29563 leader_election.cc:288] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:48.934749 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:48.934816 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:48.934870 27063 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:48.934922 29488 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:48.936064 29016 leader_election.cc:302] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:48.936352 29997 raft_consensus.cc:2781] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:48.936401 29997 raft_consensus.cc:482] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:48.936434 29997 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.937054 29997 raft_consensus.cc:504] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.937319 29997 leader_election.cc:288] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:48.937690 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:48.937785 27063 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.938164 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:48.938264 29488 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.938673 27063 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:48.939007 30819 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:48.939049 29016 leader_election.cc:302] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:48.939090 30819 ts_tablet_manager.cc:1367] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:48.939213 29488 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:48.939517 30819 raft_consensus.cc:348] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:48.939652 30819 raft_consensus.cc:374] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.939700 30819 raft_consensus.cc:729] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.939877 30819 consensus_queue.cc:260] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:48.940150 30819 ts_tablet_manager.cc:1398] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:48.940256 29997 raft_consensus.cc:2781] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:48.940351 29997 raft_consensus.cc:686] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:48.940465 29997 consensus_queue.cc:237] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.942144 26513 catalog_manager.cc:5484] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:48.942243 30783 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:48.942318 30783 ts_tablet_manager.cc:1367] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:58:48.943022 30783 raft_consensus.cc:348] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:48.943121 30783 raft_consensus.cc:374] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.943152 30783 raft_consensus.cc:729] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.943289 30783 consensus_queue.cc:260] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:48.944104 30783 ts_tablet_manager.cc:1398] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:48.944334 30783 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:48.945206 30783 tablet_bootstrap.cc:654] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:48.946545 30783 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:48.946620 30783 ts_tablet_manager.cc:1367] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:48.946938 30783 raft_consensus.cc:348] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.947016 30783 raft_consensus.cc:374] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:48.947043 30783 raft_consensus.cc:729] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:48.947170 30783 consensus_queue.cc:260] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:48.947418 30783 ts_tablet_manager.cc:1398] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:48.948334 26512 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 01:58:48.948446 26512 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 01:58:48.950580 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet b67e582a01b442f591d36621aa6cdc68
I20240502 01:58:48.950819 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 2e6a87dff021447da6224891638c039a
I20240502 01:58:48.951040 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 8c76b55d6462415d89dd0c591759a3be
I20240502 01:58:48.951076 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet b67e582a01b442f591d36621aa6cdc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.951087 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet b67e582a01b442f591d36621aa6cdc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:48.951252 30598 tablet_replica.cc:330] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.951269 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet b67e582a01b442f591d36621aa6cdc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.951383 30598 raft_consensus.cc:2229] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.951423 30496 tablet_replica.cc:330] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:48.951516 30598 raft_consensus.cc:2258] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.951557 30496 raft_consensus.cc:2229] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.951781 30496 raft_consensus.cc:2258] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.951990 30598 ts_tablet_manager.cc:1875] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.952244 30496 ts_tablet_manager.cc:1875] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.954211 30598 ts_tablet_manager.cc:1888] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.954250 30496 ts_tablet_manager.cc:1888] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.954280 30598 log.cc:1196] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/b67e582a01b442f591d36621aa6cdc68
I20240502 01:58:48.954301 30496 log.cc:1196] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/b67e582a01b442f591d36621aa6cdc68
I20240502 01:58:48.954517 30496 ts_tablet_manager.cc:1909] T b67e582a01b442f591d36621aa6cdc68 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.954528 30598 ts_tablet_manager.cc:1909] T b67e582a01b442f591d36621aa6cdc68 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:48.954722 29703 tablet_replica.cc:330] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.954859 29703 raft_consensus.cc:2229] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.954972 29703 raft_consensus.cc:2258] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.955257 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6a87dff021447da6224891638c039a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:48.955436 29704 tablet_replica.cc:330] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:48.955466 29703 ts_tablet_manager.cc:1875] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.955564 29704 raft_consensus.cc:2229] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.955674 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c76b55d6462415d89dd0c591759a3be with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.955740 29704 raft_consensus.cc:2258] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.955811 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6a87dff021447da6224891638c039a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:48.955843 30598 tablet_replica.cc:330] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.955960 30598 raft_consensus.cc:2229] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.956004 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet b67e582a01b442f591d36621aa6cdc68 (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.956060 30598 raft_consensus.cc:2258] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.956205 29704 ts_tablet_manager.cc:1875] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.956573 30598 ts_tablet_manager.cc:1875] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.957223 29703 ts_tablet_manager.cc:1888] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.957269 29703 log.cc:1196] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/b67e582a01b442f591d36621aa6cdc68
I20240502 01:58:48.957428 29703 ts_tablet_manager.cc:1909] T b67e582a01b442f591d36621aa6cdc68 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.957811 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet b67e582a01b442f591d36621aa6cdc68 (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.958261 29704 ts_tablet_manager.cc:1888] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.958321 29704 log.cc:1196] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/2e6a87dff021447da6224891638c039a
I20240502 01:58:48.958417 30598 ts_tablet_manager.cc:1888] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.958472 30598 log.cc:1196] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/8c76b55d6462415d89dd0c591759a3be
I20240502 01:58:48.958477 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c76b55d6462415d89dd0c591759a3be with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:48.958544 29704 ts_tablet_manager.cc:1909] T 2e6a87dff021447da6224891638c039a P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:48.958694 30598 ts_tablet_manager.cc:1909] T 8c76b55d6462415d89dd0c591759a3be P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:48.959259 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c76b55d6462415d89dd0c591759a3be with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.959331 30496 tablet_replica.cc:330] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:48.959386 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6a87dff021447da6224891638c039a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:48 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:48.959429 30496 raft_consensus.cc:2229] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.959510 30496 raft_consensus.cc:2258] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.959569 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet b67e582a01b442f591d36621aa6cdc68 (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.959769 29703 tablet_replica.cc:330] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.959883 29703 raft_consensus.cc:2229] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:48.959937 30496 ts_tablet_manager.cc:1875] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.960026 29703 raft_consensus.cc:2258] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.960402 29703 ts_tablet_manager.cc:1875] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.961710 30496 ts_tablet_manager.cc:1888] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.961769 30496 log.cc:1196] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/8c76b55d6462415d89dd0c591759a3be
I20240502 01:58:48.961964 29703 ts_tablet_manager.cc:1888] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.962016 29703 log.cc:1196] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/8c76b55d6462415d89dd0c591759a3be
I20240502 01:58:48.962249 29703 ts_tablet_manager.cc:1909] T 8c76b55d6462415d89dd0c591759a3be P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.962007 30496 ts_tablet_manager.cc:1909] T 8c76b55d6462415d89dd0c591759a3be P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:48.963171 29703 tablet_replica.cc:330] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:48.963311 29703 raft_consensus.cc:2229] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.963428 29703 raft_consensus.cc:2258] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.963706 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 8c76b55d6462415d89dd0c591759a3be (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.963835 29703 ts_tablet_manager.cc:1875] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.963847 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 8c76b55d6462415d89dd0c591759a3be (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.963974 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 2e6a87dff021447da6224891638c039a (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.965441 29563 raft_consensus.cc:482] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:48.965454 29703 ts_tablet_manager.cc:1888] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.965515 29703 log.cc:1196] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/2e6a87dff021447da6224891638c039a
I20240502 01:58:48.965523 29563 raft_consensus.cc:504] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.965742 29703 ts_tablet_manager.cc:1909] T 2e6a87dff021447da6224891638c039a P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:48.965859 29563 leader_election.cc:288] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:48.966300 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751e68570b084b8382751ee3e1cfb727" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:48.966439 29349 raft_consensus.cc:2454] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:48.966717 29013 leader_election.cc:302] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:48.966850 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 2e6a87dff021447da6224891638c039a (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.966881 29563 raft_consensus.cc:2781] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:48.966926 29563 raft_consensus.cc:482] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:48.966958 29563 raft_consensus.cc:3037] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.967038 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751e68570b084b8382751ee3e1cfb727" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:48.967165 29488 raft_consensus.cc:2454] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:48.967824 29563 raft_consensus.cc:504] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.968101 29563 leader_election.cc:288] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:48.968384 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751e68570b084b8382751ee3e1cfb727" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:48.968494 29349 raft_consensus.cc:3037] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.968484 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751e68570b084b8382751ee3e1cfb727" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:48.968590 29488 raft_consensus.cc:3037] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:48.968647 30598 tablet_replica.cc:330] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:48.968730 30598 raft_consensus.cc:2229] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:48.968809 30598 raft_consensus.cc:2258] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:48.969115 30598 ts_tablet_manager.cc:1875] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:48.969405 29488 raft_consensus.cc:2454] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:48.969406 29349 raft_consensus.cc:2454] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:48.969715 29013 leader_election.cc:302] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:48.969857 29563 raft_consensus.cc:2781] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:48.969950 29563 raft_consensus.cc:686] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:48.970084 29563 consensus_queue.cc:237] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:48.970316 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 8c76b55d6462415d89dd0c591759a3be (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.971601 26514 catalog_manager.cc:5484] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:48.978613 30598 ts_tablet_manager.cc:1888] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:48.978678 30598 log.cc:1196] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/2e6a87dff021447da6224891638c039a
I20240502 01:58:48.978942 30598 ts_tablet_manager.cc:1909] T 2e6a87dff021447da6224891638c039a P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:48.979526 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 2e6a87dff021447da6224891638c039a (table rebalance_test_table_10 [id=5d5338285c53470eb5116a351534a54a]) successfully deleted
I20240502 01:58:48.992141 27042 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:49.001415 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:49.002166 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:49.010617 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:49.011234 26775 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:49.013274 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:49.024139 26639 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:49.026356 29058 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:49.071573 27063 consensus_queue.cc:237] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.072731 26930 raft_consensus.cc:1261] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.072844 29349 raft_consensus.cc:1261] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.073127 30680 consensus_queue.cc:1035] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.073334 29625 consensus_queue.cc:1035] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.075006 30680 raft_consensus.cc:2932] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.075548 26930 raft_consensus.cc:2932] T 7f5513cf299e44038525ad5f438d74ec P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.076678 29349 raft_consensus.cc:2932] T 7f5513cf299e44038525ad5f438d74ec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.078382 26514 catalog_manager.cc:5484] T 7f5513cf299e44038525ad5f438d74ec P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.080205 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:49.081849 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f5513cf299e44038525ad5f438d74ec with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:49.082095 30496 tablet_replica.cc:330] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:49.082249 30496 raft_consensus.cc:2229] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.082401 30496 raft_consensus.cc:2258] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.082924 30496 ts_tablet_manager.cc:1875] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:49.084949 29349 raft_consensus.cc:1261] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:49.085096 30496 ts_tablet_manager.cc:1888] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:49.085160 30496 log.cc:1196] T 7f5513cf299e44038525ad5f438d74ec P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/7f5513cf299e44038525ad5f438d74ec
I20240502 01:58:49.085186 29488 raft_consensus.cc:1261] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:49.085317 29997 consensus_queue.cc:1035] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.085813 29997 consensus_queue.cc:1035] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.085919 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 7f5513cf299e44038525ad5f438d74ec (table rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]) successfully deleted
I20240502 01:58:49.087934 26513 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 751e68570b084b8382751ee3e1cfb727 (table rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270]). Expected version 1 got 0
I20240502 01:58:49.088306 26511 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 751e68570b084b8382751ee3e1cfb727 (table rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270]). Expected version 1 got 0
I20240502 01:58:49.089093 30870 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:49.089452 30870 tablet.cc:1620] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:49.089582 30440 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:49.089846 30440 tablet.cc:1620] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:49.092875 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270] alter complete (version 1)
I20240502 01:58:49.093078 26511 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 751e68570b084b8382751ee3e1cfb727 (table rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270]). Expected version 1 got 0
I20240502 01:58:49.093075 30871 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:49.093652 30871 tablet.cc:1620] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:49.120718 29563 raft_consensus.cc:482] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:49.120837 29563 raft_consensus.cc:504] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.121220 29563 leader_election.cc:288] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:49.121476 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2563da5db7a4e5197532b31e9aef636" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:49.121600 29488 raft_consensus.cc:2454] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:49.121603 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2563da5db7a4e5197532b31e9aef636" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:49.121726 29213 raft_consensus.cc:2454] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:49.121871 29016 leader_election.cc:302] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:49.122014 29563 raft_consensus.cc:2781] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:49.122108 29563 raft_consensus.cc:482] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:49.122146 29563 raft_consensus.cc:3037] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.123054 29563 raft_consensus.cc:504] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.123366 29563 leader_election.cc:288] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:49.123641 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2563da5db7a4e5197532b31e9aef636" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:49.123739 29488 raft_consensus.cc:3037] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.123813 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2563da5db7a4e5197532b31e9aef636" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:49.123926 29213 raft_consensus.cc:3037] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.124603 29488 raft_consensus.cc:2454] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:49.124890 29213 raft_consensus.cc:2454] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:49.125559 29015 leader_election.cc:302] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:49.125715 29563 raft_consensus.cc:2781] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:49.125785 29563 raft_consensus.cc:686] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:49.125902 29563 consensus_queue.cc:237] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.128741 26511 catalog_manager.cc:5484] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:49.195562 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:49.200143 30870 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:49.200449 30870 tablet.cc:1620] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:49.200606 30871 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:49.200661 30440 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:49.200892 30871 tablet.cc:1620] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:49.200958 30440 tablet.cc:1620] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:49.205380 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=e4f15958268847269819a9a502db5270] alter complete (version 2)
I20240502 01:58:49.267675 30909 raft_consensus.cc:482] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:49.267765 30909 raft_consensus.cc:504] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:49.268194 30909 leader_election.cc:288] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:49.268476 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:49.268607 29488 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:49.268857 29016 leader_election.cc:302] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:49.269011 30909 raft_consensus.cc:2781] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:49.269057 30909 raft_consensus.cc:482] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:49.269091 30909 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.269820 30909 raft_consensus.cc:504] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:49.270196 30909 leader_election.cc:288] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:49.270453 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:49.270562 29488 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.271476 29488 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:49.271742 29016 leader_election.cc:302] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:49.271916 30909 raft_consensus.cc:2781] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:49.272003 30909 raft_consensus.cc:686] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:49.272132 30909 consensus_queue.cc:237] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:49.272218 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:49.272372 26661 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.273298 26661 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:49.272202 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:49.273732 26658 raft_consensus.cc:2362] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1. Re-sending same reply.
I20240502 01:58:49.274111 26513 catalog_manager.cc:5484] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:49.298338 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:49.300405 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:49.309231 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:49.311128 26639 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:49.311882 26775 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:49.322760 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:49.322993 29058 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:49.323765 30870 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:49.324180 27042 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:58:49.329500 30440 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:49.331881 30871 tablet.cc:1722] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:49.335289 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=e4f15958268847269819a9a502db5270] alter complete (version 3)
I20240502 01:58:49.342715 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:49.343632 30909 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.352051 29563 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.396760 27064 consensus_queue.cc:237] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.398073 26930 raft_consensus.cc:1261] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.398085 29349 raft_consensus.cc:1261] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.398420 30680 consensus_queue.cc:1035] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.398586 29625 consensus_queue.cc:1035] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.400213 28915 raft_consensus.cc:2932] T cceb8a91997e4db9a4842df26ad3b02e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.400653 29349 raft_consensus.cc:2932] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.400717 26930 raft_consensus.cc:2932] T cceb8a91997e4db9a4842df26ad3b02e P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.403385 26511 catalog_manager.cc:5484] T cceb8a91997e4db9a4842df26ad3b02e P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.405580 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet cceb8a91997e4db9a4842df26ad3b02e with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:49.405790 30496 tablet_replica.cc:330] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:49.405932 30496 raft_consensus.cc:2229] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.406080 30496 raft_consensus.cc:2258] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.406584 30496 ts_tablet_manager.cc:1875] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:49.408576 30496 ts_tablet_manager.cc:1888] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:49.408636 30496 log.cc:1196] T cceb8a91997e4db9a4842df26ad3b02e P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/cceb8a91997e4db9a4842df26ad3b02e
I20240502 01:58:49.409094 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet cceb8a91997e4db9a4842df26ad3b02e (table rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]) successfully deleted
I20240502 01:58:49.442494 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:49.442615 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:49.444504 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 751e68570b084b8382751ee3e1cfb727
I20240502 01:58:49.444960 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 751e68570b084b8382751ee3e1cfb727 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:49.445380 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet 751e68570b084b8382751ee3e1cfb727 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:49.445919 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 751e68570b084b8382751ee3e1cfb727 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:49.446125 30989 tablet_replica.cc:330] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:49.446300 30989 raft_consensus.cc:2229] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.446475 30989 raft_consensus.cc:2258] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.447103 30989 ts_tablet_manager.cc:1875] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.447455 30988 tablet_replica.cc:330] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:49.447585 30988 raft_consensus.cc:2229] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.447932 30988 raft_consensus.cc:2258] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.448426 30990 tablet_replica.cc:330] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:49.448931 30990 raft_consensus.cc:2229] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.449290 30988 ts_tablet_manager.cc:1875] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.450142 30989 ts_tablet_manager.cc:1888] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.450222 30989 log.cc:1196] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/751e68570b084b8382751ee3e1cfb727
I20240502 01:58:49.449849 30990 raft_consensus.cc:2258] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.452215 30989 ts_tablet_manager.cc:1909] T 751e68570b084b8382751ee3e1cfb727 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:49.452832 30990 ts_tablet_manager.cc:1875] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.452879 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 751e68570b084b8382751ee3e1cfb727 (table rebalancer_dynamic_table_new_name [id=e4f15958268847269819a9a502db5270]) successfully deleted
I20240502 01:58:49.455680 30988 ts_tablet_manager.cc:1888] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.455771 30988 log.cc:1196] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/751e68570b084b8382751ee3e1cfb727
I20240502 01:58:49.457093 30988 ts_tablet_manager.cc:1909] T 751e68570b084b8382751ee3e1cfb727 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:49.457559 30990 ts_tablet_manager.cc:1888] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.457636 30990 log.cc:1196] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/751e68570b084b8382751ee3e1cfb727
I20240502 01:58:49.457669 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 751e68570b084b8382751ee3e1cfb727 (table rebalancer_dynamic_table_new_name [id=e4f15958268847269819a9a502db5270]) successfully deleted
I20240502 01:58:49.457862 30990 ts_tablet_manager.cc:1909] T 751e68570b084b8382751ee3e1cfb727 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:49.458122 26511 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 01:58:49.458209 26511 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 01:58:49.458544 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 751e68570b084b8382751ee3e1cfb727 (table rebalancer_dynamic_table_new_name [id=e4f15958268847269819a9a502db5270]) successfully deleted
I20240502 01:58:49.460412 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 2adcab4321d549919938465f639c9ddc
I20240502 01:58:49.460620 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 8d1f07f16b0f49b4b3c3cc5511604532
I20240502 01:58:49.460819 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet 2adcab4321d549919938465f639c9ddc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:49.460846 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet 1417ea39bb2b4716bfb3d11c05853c03
I20240502 01:58:49.460915 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 2adcab4321d549919938465f639c9ddc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:49.461001 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 2adcab4321d549919938465f639c9ddc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:49.461170 29704 tablet_replica.cc:330] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:49.461218 29703 tablet_replica.cc:330] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:49.461316 29704 raft_consensus.cc:2229] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.461341 29703 raft_consensus.cc:2229] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.461395 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:49.461441 29704 raft_consensus.cc:2258] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.461517 30990 tablet_replica.cc:330] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:49.461526 29703 raft_consensus.cc:2258] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.461616 30990 raft_consensus.cc:2229] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.461688 30990 raft_consensus.cc:2258] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.461906 29704 ts_tablet_manager.cc:1875] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.462133 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1f07f16b0f49b4b3c3cc5511604532 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:49.462148 30496 tablet_replica.cc:330] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:49.462221 30990 ts_tablet_manager.cc:1875] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.462284 30496 raft_consensus.cc:2229] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.462425 30496 raft_consensus.cc:2258] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.462916 30496 ts_tablet_manager.cc:1875] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.463589 29704 ts_tablet_manager.cc:1888] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.463652 29704 log.cc:1196] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/2adcab4321d549919938465f639c9ddc
I20240502 01:58:49.463863 29704 ts_tablet_manager.cc:1909] T 2adcab4321d549919938465f639c9ddc P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:49.463928 30990 ts_tablet_manager.cc:1888] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.464002 30990 log.cc:1196] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/1417ea39bb2b4716bfb3d11c05853c03
I20240502 01:58:49.464236 30990 ts_tablet_manager.cc:1909] T 1417ea39bb2b4716bfb3d11c05853c03 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:49.464815 30598 tablet_replica.cc:330] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:49.464947 30496 ts_tablet_manager.cc:1888] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.464954 30598 raft_consensus.cc:2229] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.465016 30496 log.cc:1196] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/2adcab4321d549919938465f639c9ddc
I20240502 01:58:49.465090 30598 raft_consensus.cc:2258] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.465250 30496 ts_tablet_manager.cc:1909] T 2adcab4321d549919938465f639c9ddc P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:49.465603 30598 ts_tablet_manager.cc:1875] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.466513 29703 ts_tablet_manager.cc:1875] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.466559 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:49.467680 30598 ts_tablet_manager.cc:1888] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.467742 30598 log.cc:1196] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/8d1f07f16b0f49b4b3c3cc5511604532
I20240502 01:58:49.467969 30598 ts_tablet_manager.cc:1909] T 8d1f07f16b0f49b4b3c3cc5511604532 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:49.468181 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 1417ea39bb2b4716bfb3d11c05853c03 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.468564 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:49.468626 29703 ts_tablet_manager.cc:1888] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.468691 29703 log.cc:1196] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/2adcab4321d549919938465f639c9ddc
I20240502 01:58:49.468708 29704 tablet_replica.cc:330] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:49.468752 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 2adcab4321d549919938465f639c9ddc (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.468822 29704 raft_consensus.cc:2229] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.468868 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1f07f16b0f49b4b3c3cc5511604532 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:49.468981 29704 raft_consensus.cc:2258] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.469048 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1f07f16b0f49b4b3c3cc5511604532 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:49.469130 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 2adcab4321d549919938465f639c9ddc (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.469136 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 1417ea39bb2b4716bfb3d11c05853c03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:49.469362 29703 ts_tablet_manager.cc:1909] T 2adcab4321d549919938465f639c9ddc P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:49.469405 29704 ts_tablet_manager.cc:1875] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.469659 30598 tablet_replica.cc:330] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:49.469761 30598 raft_consensus.cc:2229] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.469756 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 8d1f07f16b0f49b4b3c3cc5511604532 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.469874 30598 raft_consensus.cc:2258] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.469923 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 2adcab4321d549919938465f639c9ddc (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.470098 30496 tablet_replica.cc:330] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:49.470217 30496 raft_consensus.cc:2229] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.470348 30496 raft_consensus.cc:2258] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.470372 30598 ts_tablet_manager.cc:1875] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.470779 30496 ts_tablet_manager.cc:1875] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.471210 29704 ts_tablet_manager.cc:1888] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.471273 29704 log.cc:1196] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/1417ea39bb2b4716bfb3d11c05853c03
I20240502 01:58:49.471520 29704 ts_tablet_manager.cc:1909] T 1417ea39bb2b4716bfb3d11c05853c03 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:49.472235 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 1417ea39bb2b4716bfb3d11c05853c03 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.472322 30598 ts_tablet_manager.cc:1888] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.472397 30598 log.cc:1196] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/1417ea39bb2b4716bfb3d11c05853c03
I20240502 01:58:49.472524 30496 ts_tablet_manager.cc:1888] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.472591 30496 log.cc:1196] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/8d1f07f16b0f49b4b3c3cc5511604532
I20240502 01:58:49.472635 30598 ts_tablet_manager.cc:1909] T 1417ea39bb2b4716bfb3d11c05853c03 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:49.472819 30496 ts_tablet_manager.cc:1909] T 8d1f07f16b0f49b4b3c3cc5511604532 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:49.473196 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 1417ea39bb2b4716bfb3d11c05853c03 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.473335 29703 tablet_replica.cc:330] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:49.473376 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 8d1f07f16b0f49b4b3c3cc5511604532 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.473466 29703 raft_consensus.cc:2229] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.473623 29703 raft_consensus.cc:2258] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.474059 29703 ts_tablet_manager.cc:1875] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.475819 29703 ts_tablet_manager.cc:1888] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.475921 29703 log.cc:1196] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/8d1f07f16b0f49b4b3c3cc5511604532
I20240502 01:58:49.476146 29703 ts_tablet_manager.cc:1909] T 8d1f07f16b0f49b4b3c3cc5511604532 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:49.476737 29703 tablet_replica.cc:330] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:49.476897 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 8d1f07f16b0f49b4b3c3cc5511604532 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.477120 29703 raft_consensus.cc:2229] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.477249 29703 raft_consensus.cc:2258] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.477679 29703 ts_tablet_manager.cc:1875] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.479079 29703 ts_tablet_manager.cc:1888] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:49.479144 29703 log.cc:1196] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/1417ea39bb2b4716bfb3d11c05853c03
I20240502 01:58:49.479331 29703 ts_tablet_manager.cc:1909] T 1417ea39bb2b4716bfb3d11c05853c03 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:49.479853 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 1417ea39bb2b4716bfb3d11c05853c03 (table rebalance_test_table_12 [id=ba4c83ebc427424aa2e600c40f538972]) successfully deleted
I20240502 01:58:49.509227 29563 consensus_queue.cc:1035] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.514681 29563 consensus_queue.cc:1035] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.540257 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:49.541111 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:49.542829 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:49.545075 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:49.550428 26514 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:49.563036 29058 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:49.566692 29451 tablet_service.cc:1467] Processing CreateTablet for tablet c13e8de8518a4965a82bf46edcadd996 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b]), partition=
I20240502 01:58:49.566915 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c13e8de8518a4965a82bf46edcadd996. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.567875 27042 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:58:49.569051 29196 tablet_service.cc:1467] Processing CreateTablet for tablet c13e8de8518a4965a82bf46edcadd996 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b]), partition=
I20240502 01:58:49.569247 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c13e8de8518a4965a82bf46edcadd996. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.570518 26909 tablet_service.cc:1467] Processing CreateTablet for tablet c13e8de8518a4965a82bf46edcadd996 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b]), partition=
I20240502 01:58:49.570735 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c13e8de8518a4965a82bf46edcadd996. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.572777 31046 tablet_bootstrap.cc:492] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:49.573472 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:49.574738 31046 tablet_bootstrap.cc:654] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.575757 31046 tablet_bootstrap.cc:492] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:49.575836 31046 ts_tablet_manager.cc:1367] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:49.576185 31046 raft_consensus.cc:348] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.576275 31046 raft_consensus.cc:374] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.576311 31046 raft_consensus.cc:729] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.576411 31046 consensus_queue.cc:260] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.576694 31046 ts_tablet_manager.cc:1398] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:49.583441 31045 tablet_bootstrap.cc:492] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:49.584352 31045 tablet_bootstrap.cc:654] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.586555 31044 tablet_bootstrap.cc:492] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:49.587428 31044 tablet_bootstrap.cc:654] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.591069 26775 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:49.591339 31045 tablet_bootstrap.cc:492] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:49.591434 31045 ts_tablet_manager.cc:1367] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:58:49.591871 31045 raft_consensus.cc:348] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.591977 31045 raft_consensus.cc:374] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.592010 31045 raft_consensus.cc:729] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.592140 31045 consensus_queue.cc:260] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.592438 31045 ts_tablet_manager.cc:1398] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:49.590814 31044 tablet_bootstrap.cc:492] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:49.592638 31044 ts_tablet_manager.cc:1367] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:49.593092 31044 raft_consensus.cc:348] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.593225 31044 raft_consensus.cc:374] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.593273 31044 raft_consensus.cc:729] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.593405 31044 consensus_queue.cc:260] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.593751 31044 ts_tablet_manager.cc:1398] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:49.638592 26661 consensus_queue.cc:237] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.639707 29488 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.639737 26930 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.640077 31066 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.640275 30768 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.642179 31069 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:49.642343 29488 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:49.642408 26930 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:49.644832 26514 catalog_manager.cc:5484] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:49.648115 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet e1cd4681de8645ff92b479ccdd3eac37 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:49.648332 29704 tablet_replica.cc:330] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:49.648483 29704 raft_consensus.cc:2229] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.648617 29704 raft_consensus.cc:2258] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.649104 29704 ts_tablet_manager.cc:1875] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:49.651052 29704 ts_tablet_manager.cc:1888] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:49.651127 29704 log.cc:1196] T e1cd4681de8645ff92b479ccdd3eac37 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/e1cd4681de8645ff92b479ccdd3eac37
I20240502 01:58:49.651567 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet e1cd4681de8645ff92b479ccdd3eac37 (table rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]) successfully deleted
I20240502 01:58:49.673243 30500 raft_consensus.cc:482] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:49.673327 30500 raft_consensus.cc:504] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.673650 30500 leader_election.cc:288] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:49.673940 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13e8de8518a4965a82bf46edcadd996" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:49.674055 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13e8de8518a4965a82bf46edcadd996" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:49.674106 29488 raft_consensus.cc:2454] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:49.674171 29213 raft_consensus.cc:2454] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:49.674412 26861 leader_election.cc:302] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:49.674582 30500 raft_consensus.cc:2781] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:49.674891 30500 raft_consensus.cc:482] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:49.674921 30500 raft_consensus.cc:3037] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.675760 30500 raft_consensus.cc:504] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.676028 30500 leader_election.cc:288] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:49.676331 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13e8de8518a4965a82bf46edcadd996" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:49.676436 29488 raft_consensus.cc:3037] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.676628 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c13e8de8518a4965a82bf46edcadd996" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:49.676721 29213 raft_consensus.cc:3037] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.677318 29488 raft_consensus.cc:2454] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:49.677568 29213 raft_consensus.cc:2454] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:49.677580 26862 leader_election.cc:302] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:49.677716 30500 raft_consensus.cc:2781] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:49.677875 30500 raft_consensus.cc:686] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:49.678009 30500 consensus_queue.cc:237] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:49.679922 26515 catalog_manager.cc:5484] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:49.716946 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:49.727104 27042 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:58:49.728008 29330 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:58:49.728454 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:49.729444 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:49.730751 26909 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:49.738952 26775 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:49.765653 29058 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:49.788718 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:49.794380 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 01764f37959e4c6799e3a83f0d4ed2e1 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.794518 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 01764f37959e4c6799e3a83f0d4ed2e1 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.794518 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 01764f37959e4c6799e3a83f0d4ed2e1 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.794685 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01764f37959e4c6799e3a83f0d4ed2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.794773 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01764f37959e4c6799e3a83f0d4ed2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.794773 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01764f37959e4c6799e3a83f0d4ed2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.797071 31129 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:49.797124 31045 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:49.797598 29058 tablet_service.cc:1467] Processing CreateTablet for tablet a0ea54806f7342e2bb3d2f287bb84bdc (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.797852 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ea54806f7342e2bb3d2f287bb84bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.797955 31129 tablet_bootstrap.cc:654] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.797959 31045 tablet_bootstrap.cc:654] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.798552 31128 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:49.799537 31128 tablet_bootstrap.cc:654] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.799618 29196 tablet_service.cc:1467] Processing CreateTablet for tablet a0ea54806f7342e2bb3d2f287bb84bdc (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.799849 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ea54806f7342e2bb3d2f287bb84bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.800000 31129 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:49.800081 31129 ts_tablet_manager.cc:1367] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:49.800565 31132 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:49.800616 31129 raft_consensus.cc:348] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.800714 31129 raft_consensus.cc:374] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.800741 31129 raft_consensus.cc:729] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.800843 31129 consensus_queue.cc:260] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.801147 31129 ts_tablet_manager.cc:1398] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:49.802145 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 734e4d2d6ca746dabc297b73c7744ca8 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.802271 31128 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:49.802331 31128 ts_tablet_manager.cc:1367] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:49.802332 31132 tablet_bootstrap.cc:654] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.802390 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734e4d2d6ca746dabc297b73c7744ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.802639 31128 raft_consensus.cc:348] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.802709 31128 raft_consensus.cc:374] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.802733 31128 raft_consensus.cc:729] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.802856 31128 consensus_queue.cc:260] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.803095 31128 ts_tablet_manager.cc:1398] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:49.804901 26639 tablet_service.cc:1467] Processing CreateTablet for tablet a0ea54806f7342e2bb3d2f287bb84bdc (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.805097 26639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ea54806f7342e2bb3d2f287bb84bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.804953 31139 tablet_bootstrap.cc:492] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:49.806311 31139 tablet_bootstrap.cc:654] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.806952 31129 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:49.807705 31129 tablet_bootstrap.cc:654] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.808182 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 734e4d2d6ca746dabc297b73c7744ca8 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.808378 30909 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.808447 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734e4d2d6ca746dabc297b73c7744ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.808677 31129 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:49.808748 31129 ts_tablet_manager.cc:1367] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:49.809234 31129 raft_consensus.cc:348] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.809337 31129 raft_consensus.cc:374] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.809366 31129 raft_consensus.cc:729] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.809480 31129 consensus_queue.cc:260] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.809736 31129 ts_tablet_manager.cc:1398] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:49.810313 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:49.810626 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 734e4d2d6ca746dabc297b73c7744ca8 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:49.810907 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 734e4d2d6ca746dabc297b73c7744ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:49.811146 31044 tablet_bootstrap.cc:492] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:49.812057 31044 tablet_bootstrap.cc:654] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.812860 31129 tablet_bootstrap.cc:492] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:49.813552 31045 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:49.813612 31129 tablet_bootstrap.cc:654] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.813644 31045 ts_tablet_manager.cc:1367] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20240502 01:58:49.814527 31045 raft_consensus.cc:348] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.814635 31045 raft_consensus.cc:374] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.814666 31045 raft_consensus.cc:729] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.814777 31045 consensus_queue.cc:260] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.815028 31045 ts_tablet_manager.cc:1398] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:49.815193 31045 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:49.816116 31045 tablet_bootstrap.cc:654] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:49.817845 29488 raft_consensus.cc:1261] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:49.818404 31086 consensus_queue.cc:1035] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.818503 29213 raft_consensus.cc:1261] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:49.818984 31086 consensus_queue.cc:1035] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.821717 26514 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet c13e8de8518a4965a82bf46edcadd996 (table rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b]). Expected version 1 got 0
I20240502 01:58:49.822618 31132 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:49.822718 31132 ts_tablet_manager.cc:1367] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240502 01:58:49.823064 26514 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet c13e8de8518a4965a82bf46edcadd996 (table rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b]). Expected version 1 got 0
I20240502 01:58:49.823155 31132 raft_consensus.cc:348] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.823247 31132 raft_consensus.cc:374] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.823287 31132 raft_consensus.cc:729] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.823391 31132 consensus_queue.cc:260] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.823680 31132 ts_tablet_manager.cc:1398] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:49.823760 30871 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:49.824013 30871 tablet.cc:1620] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:49.824677 31045 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:49.824765 31045 ts_tablet_manager.cc:1367] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240502 01:58:49.825203 31045 raft_consensus.cc:348] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.825294 31045 raft_consensus.cc:374] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.825325 31045 raft_consensus.cc:729] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.825418 31045 consensus_queue.cc:260] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.825676 31045 ts_tablet_manager.cc:1398] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:49.825954 29992 raft_consensus.cc:482] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:49.826083 29992 raft_consensus.cc:504] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.826246 31044 tablet_bootstrap.cc:492] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:49.826319 31044 ts_tablet_manager.cc:1367] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:58:49.826484 29992 leader_election.cc:288] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:49.826741 31044 raft_consensus.cc:348] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.826834 31044 raft_consensus.cc:374] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.826860 31044 raft_consensus.cc:729] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.826983 31044 consensus_queue.cc:260] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.827266 31044 ts_tablet_manager.cc:1398] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:49.827239 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01764f37959e4c6799e3a83f0d4ed2e1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:49.827353 29213 raft_consensus.cc:2454] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:49.827898 29285 leader_election.cc:302] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:49.828753 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b] alter complete (version 1)
I20240502 01:58:49.830736 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01764f37959e4c6799e3a83f0d4ed2e1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:49.830873 26661 raft_consensus.cc:2454] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:49.831032 29563 raft_consensus.cc:482] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:49.831121 29563 raft_consensus.cc:504] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.831434 29563 leader_election.cc:288] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:49.831511 29563 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:49.832244 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ea54806f7342e2bb3d2f287bb84bdc" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:49.832346 29216 raft_consensus.cc:2454] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:49.832600 29015 leader_election.cc:302] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:49.834681 31152 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:49.835661 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ea54806f7342e2bb3d2f287bb84bdc" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:49.835772 26658 raft_consensus.cc:2454] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:49.836067 30909 raft_consensus.cc:2781] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:49.836118 30909 raft_consensus.cc:482] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:49.836151 30909 raft_consensus.cc:3037] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.836134 26514 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet c13e8de8518a4965a82bf46edcadd996 (table rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b]). Expected version 1 got 0
I20240502 01:58:49.836532 31139 tablet_bootstrap.cc:492] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:49.836616 31139 ts_tablet_manager.cc:1367] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 01:58:49.837010 30909 raft_consensus.cc:504] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.837028 31139 raft_consensus.cc:348] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.837116 31139 raft_consensus.cc:374] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.837141 31139 raft_consensus.cc:729] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.837252 31139 consensus_queue.cc:260] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.837319 30909 leader_election.cc:288] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:49.835567 31152 tablet.cc:1620] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:49.837548 31139 ts_tablet_manager.cc:1398] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:49.837885 29992 raft_consensus.cc:2781] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:49.837935 29992 raft_consensus.cc:482] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:49.837968 29992 raft_consensus.cc:3037] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.838008 31129 tablet_bootstrap.cc:492] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:49.838109 31129 ts_tablet_manager.cc:1367] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.002s
I20240502 01:58:49.838516 31129 raft_consensus.cc:348] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.838606 31129 raft_consensus.cc:374] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:49.838636 31129 raft_consensus.cc:729] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:49.838747 31129 consensus_queue.cc:260] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.838908 29992 raft_consensus.cc:504] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.839035 31129 ts_tablet_manager.cc:1398] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:49.839226 29992 leader_election.cc:288] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:49.839685 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01764f37959e4c6799e3a83f0d4ed2e1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:49.839798 29216 raft_consensus.cc:3037] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.840730 29216 raft_consensus.cc:2454] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:49.841392 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ea54806f7342e2bb3d2f287bb84bdc" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:49.841487 26661 raft_consensus.cc:3037] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.842182 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01764f37959e4c6799e3a83f0d4ed2e1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:49.842280 26658 raft_consensus.cc:3037] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.842950 26658 raft_consensus.cc:2454] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:49.843178 29285 leader_election.cc:302] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:49.843451 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ea54806f7342e2bb3d2f287bb84bdc" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:49.843538 29213 raft_consensus.cc:3037] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.844138 29213 raft_consensus.cc:2454] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:49.844396 29015 leader_election.cc:302] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:49.844555 29563 raft_consensus.cc:2781] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:49.844622 29563 raft_consensus.cc:686] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:49.844744 29563 consensus_queue.cc:237] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.845115 26661 raft_consensus.cc:2454] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:49.845467 29992 raft_consensus.cc:2781] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:49.845618 29992 raft_consensus.cc:686] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:49.845738 29992 consensus_queue.cc:237] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:49.846588 26514 catalog_manager.cc:5484] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:49.847467 26513 catalog_manager.cc:5484] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:49.849179 31150 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:49.849479 31150 tablet.cc:1620] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:49.855633 26794 consensus_queue.cc:237] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:49.856736 29080 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.856787 27064 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:49.857071 30910 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.857252 30924 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:49.858822 30608 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:49.859328 27064 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:49.860205 29080 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:49.861912 26509 catalog_manager.cc:5484] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:49.864997 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 565aab51035c4f328e49eca53eddf621 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:49.865175 30598 tablet_replica.cc:330] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:49.865316 30598 raft_consensus.cc:2229] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.865440 30598 raft_consensus.cc:2258] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.865856 30598 ts_tablet_manager.cc:1875] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:49.867657 30598 ts_tablet_manager.cc:1888] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:49.867722 30598 log.cc:1196] T 565aab51035c4f328e49eca53eddf621 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/565aab51035c4f328e49eca53eddf621
I20240502 01:58:49.868146 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 565aab51035c4f328e49eca53eddf621 (table rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]) successfully deleted
I20240502 01:58:49.914758 31066 raft_consensus.cc:482] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:49.914863 31066 raft_consensus.cc:504] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.915302 31066 leader_election.cc:288] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:49.915817 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734e4d2d6ca746dabc297b73c7744ca8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:49.915944 29488 raft_consensus.cc:2454] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:49.916154 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734e4d2d6ca746dabc297b73c7744ca8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:49.916276 26794 raft_consensus.cc:2454] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:49.916558 26595 leader_election.cc:302] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:49.916728 31066 raft_consensus.cc:2781] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:49.916782 31066 raft_consensus.cc:482] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:49.916811 31066 raft_consensus.cc:3037] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.917866 31066 raft_consensus.cc:504] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.918279 31066 leader_election.cc:288] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:49.918627 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734e4d2d6ca746dabc297b73c7744ca8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:49.918727 29488 raft_consensus.cc:3037] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.919131 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "734e4d2d6ca746dabc297b73c7744ca8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:49.919247 26794 raft_consensus.cc:3037] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:49.919705 29488 raft_consensus.cc:2454] T 734e4d2d6ca746dabc297b73c7744ca8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:49.920150 26794 raft_consensus.cc:2454] T 734e4d2d6ca746dabc297b73c7744ca8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:49.920862 26594 leader_election.cc:302] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:49.922442 31066 raft_consensus.cc:2781] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:49.922523 31066 raft_consensus.cc:686] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:49.922636 31066 consensus_queue.cc:237] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:49.924707 26513 catalog_manager.cc:5484] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:49.931242 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:49.937590 31150 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:49.937896 31150 tablet.cc:1620] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:49.938076 30871 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:49.938340 30871 tablet.cc:1620] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:49.938616 31152 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:49.938891 31152 tablet.cc:1620] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:49.943054 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=d6e0c8ba2359452cafca16293bd2a10b] alter complete (version 2)
I20240502 01:58:49.954527 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:49.956244 27042 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:58:49.959451 26641 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:49.967206 26513 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 01:58:49.967314 26513 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 01:58:49.967393 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:49.969669 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 26ac02baf94743f1b6068748d83f0fe8
I20240502 01:58:49.969951 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet e60816eacfb547cda5a6c87c68bf101d
I20240502 01:58:49.970146 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 9bad5aeb22544e40b601ce3a216f89c7
I20240502 01:58:49.972108 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet e60816eacfb547cda5a6c87c68bf101d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:49.972268 29704 tablet_replica.cc:330] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:49.972399 29704 raft_consensus.cc:2229] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.972574 29704 raft_consensus.cc:2258] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.973069 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac02baf94743f1b6068748d83f0fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:49.973102 29704 ts_tablet_manager.cc:1875] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.973206 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bad5aeb22544e40b601ce3a216f89c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:49.973327 30496 tablet_replica.cc:330] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:49.973439 30496 raft_consensus.cc:2229] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.973546 30496 raft_consensus.cc:2258] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.973958 30496 ts_tablet_manager.cc:1875] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.974375 26639 tablet_service.cc:1514] Processing DeleteTablet for tablet e60816eacfb547cda5a6c87c68bf101d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:49.974539 29703 tablet_replica.cc:330] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:49.974663 29703 raft_consensus.cc:2229] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.974704 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac02baf94743f1b6068748d83f0fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:49.974807 29703 raft_consensus.cc:2258] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.974855 30598 tablet_replica.cc:330] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:49.974956 30598 raft_consensus.cc:2229] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.975116 30598 raft_consensus.cc:2258] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.975294 29703 ts_tablet_manager.cc:1875] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.975359 29704 ts_tablet_manager.cc:1888] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:49.975419 29704 log.cc:1196] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/e60816eacfb547cda5a6c87c68bf101d
I20240502 01:58:49.975545 30598 ts_tablet_manager.cc:1875] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.975639 29704 ts_tablet_manager.cc:1909] T e60816eacfb547cda5a6c87c68bf101d P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:49.976313 29058 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:49.976483 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bad5aeb22544e40b601ce3a216f89c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:49.977190 30496 ts_tablet_manager.cc:1888] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.977247 30496 log.cc:1196] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/26ac02baf94743f1b6068748d83f0fe8
I20240502 01:58:49.977401 29703 ts_tablet_manager.cc:1888] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:49.977463 29703 log.cc:1196] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/e60816eacfb547cda5a6c87c68bf101d
I20240502 01:58:49.977480 30496 ts_tablet_manager.cc:1909] T 26ac02baf94743f1b6068748d83f0fe8 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:49.977497 30598 ts_tablet_manager.cc:1888] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.977552 30598 log.cc:1196] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/26ac02baf94743f1b6068748d83f0fe8
I20240502 01:58:49.977676 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bad5aeb22544e40b601ce3a216f89c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:49.977768 30598 ts_tablet_manager.cc:1909] T 26ac02baf94743f1b6068748d83f0fe8 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:49.977978 30496 tablet_replica.cc:330] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:49.978183 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet e60816eacfb547cda5a6c87c68bf101d (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:49.978488 30598 tablet_replica.cc:330] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:49.978590 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac02baf94743f1b6068748d83f0fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:49.978607 30598 raft_consensus.cc:2229] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.978601 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 26ac02baf94743f1b6068748d83f0fe8 (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:49.978719 30598 raft_consensus.cc:2258] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.978857 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 26ac02baf94743f1b6068748d83f0fe8 (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:49.979306 30496 raft_consensus.cc:2229] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:49.979377 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet e60816eacfb547cda5a6c87c68bf101d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:49 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:49.979483 30496 raft_consensus.cc:2258] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.979853 30598 ts_tablet_manager.cc:1875] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.979892 30496 ts_tablet_manager.cc:1875] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:49.980145 29703 ts_tablet_manager.cc:1909] T e60816eacfb547cda5a6c87c68bf101d P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:49.980734 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet e60816eacfb547cda5a6c87c68bf101d (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:49.980839 29703 tablet_replica.cc:330] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:49.980984 29703 raft_consensus.cc:2229] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:49.981220 29703 raft_consensus.cc:2258] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:49.981792 30496 ts_tablet_manager.cc:1888] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.981905 30496 log.cc:1196] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/9bad5aeb22544e40b601ce3a216f89c7
I20240502 01:58:49.981959 30598 ts_tablet_manager.cc:1888] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:49.982050 30598 log.cc:1196] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/9bad5aeb22544e40b601ce3a216f89c7
I20240502 01:58:49.982254 30496 ts_tablet_manager.cc:1909] T 9bad5aeb22544e40b601ce3a216f89c7 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:49.982391 30598 ts_tablet_manager.cc:1909] T 9bad5aeb22544e40b601ce3a216f89c7 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:49.982969 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 9bad5aeb22544e40b601ce3a216f89c7 (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:49.988859 26907 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:49.989265 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 9bad5aeb22544e40b601ce3a216f89c7 (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:49.989869 29703 ts_tablet_manager.cc:1875] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.000279 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.001459 29703 ts_tablet_manager.cc:1888] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:50.010128 29703 log.cc:1196] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/9bad5aeb22544e40b601ce3a216f89c7
I20240502 01:58:50.010417 29703 ts_tablet_manager.cc:1909] T 9bad5aeb22544e40b601ce3a216f89c7 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:50.010967 29703 tablet_replica.cc:330] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:50.011050 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 9bad5aeb22544e40b601ce3a216f89c7 (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:50.011099 29703 raft_consensus.cc:2229] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.011227 29703 raft_consensus.cc:2258] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.011713 29703 ts_tablet_manager.cc:1875] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.017969 29703 ts_tablet_manager.cc:1888] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:50.018064 29703 log.cc:1196] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/26ac02baf94743f1b6068748d83f0fe8
I20240502 01:58:50.018522 29703 ts_tablet_manager.cc:1909] T 26ac02baf94743f1b6068748d83f0fe8 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:50.019357 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 26ac02baf94743f1b6068748d83f0fe8 (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:50.019368 31222 tablet_replica.cc:330] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:50.019922 31222 raft_consensus.cc:2229] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.019939 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.020635 31222 raft_consensus.cc:2258] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.021333 31222 ts_tablet_manager.cc:1875] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.023063 31222 ts_tablet_manager.cc:1888] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:50.023128 31222 log.cc:1196] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/e60816eacfb547cda5a6c87c68bf101d
I20240502 01:58:50.023409 31222 ts_tablet_manager.cc:1909] T e60816eacfb547cda5a6c87c68bf101d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:50.024067 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet e60816eacfb547cda5a6c87c68bf101d (table rebalance_test_table_14 [id=1289082436504a27bf6970f9b81fb4ef]) successfully deleted
I20240502 01:58:50.045572 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:50.050557 31150 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:50.051092 31152 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:50.051568 30871 tablet.cc:1722] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:50.055848 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=d6e0c8ba2359452cafca16293bd2a10b] alter complete (version 3)
I20240502 01:58:50.061769 26794 consensus_queue.cc:237] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.062976 29213 raft_consensus.cc:1261] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.063004 26661 raft_consensus.cc:1261] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.063351 30924 consensus_queue.cc:1035] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.063531 30608 consensus_queue.cc:1035] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.065104 30910 raft_consensus.cc:2932] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.065547 26661 raft_consensus.cc:2932] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.065533 29213 raft_consensus.cc:2932] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.067909 26507 catalog_manager.cc:5484] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.070346 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:50.070545 30598 tablet_replica.cc:330] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:50.070694 30598 raft_consensus.cc:2229] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.070832 30598 raft_consensus.cc:2258] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.071329 30598 ts_tablet_manager.cc:1875] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:50.073334 30598 ts_tablet_manager.cc:1888] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:50.073393 30598 log.cc:1196] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/d3e2d17af0194d5d8c08ad8372b1e9c1
I20240502 01:58:50.074394 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:50.144930 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.145243 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:50.149185 26907 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:50.150388 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.158334 26506 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:50.158465 26506 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:50.162266 27042 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.163560 26506 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet c13e8de8518a4965a82bf46edcadd996
I20240502 01:58:50.164937 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet c13e8de8518a4965a82bf46edcadd996 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:50.165141 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet c13e8de8518a4965a82bf46edcadd996 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:50.165174 31222 tablet_replica.cc:330] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:50.165284 30598 tablet_replica.cc:330] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:50.165303 31222 raft_consensus.cc:2229] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.165406 31222 raft_consensus.cc:2258] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.165400 30598 raft_consensus.cc:2229] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:50.165575 30598 raft_consensus.cc:2258] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.166090 31222 ts_tablet_manager.cc:1875] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.166496 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet c13e8de8518a4965a82bf46edcadd996 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:50.167327 30598 ts_tablet_manager.cc:1875] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.168176 31222 ts_tablet_manager.cc:1888] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.168236 31222 log.cc:1196] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/c13e8de8518a4965a82bf46edcadd996
I20240502 01:58:50.168470 31222 ts_tablet_manager.cc:1909] T c13e8de8518a4965a82bf46edcadd996 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:50.169296 30598 ts_tablet_manager.cc:1888] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.169351 30598 log.cc:1196] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/c13e8de8518a4965a82bf46edcadd996
I20240502 01:58:50.169376 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet c13e8de8518a4965a82bf46edcadd996 (table rebalancer_dynamic_table_new_name [id=d6e0c8ba2359452cafca16293bd2a10b]) successfully deleted
I20240502 01:58:50.169567 30598 ts_tablet_manager.cc:1909] T c13e8de8518a4965a82bf46edcadd996 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:50.171242 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet c13e8de8518a4965a82bf46edcadd996 (table rebalancer_dynamic_table_new_name [id=d6e0c8ba2359452cafca16293bd2a10b]) successfully deleted
I20240502 01:58:50.171766 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:50.174106 31291 tablet_replica.cc:330] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:50.174254 31291 raft_consensus.cc:2229] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.174388 31291 raft_consensus.cc:2258] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.175873 29058 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:50.184304 31291 ts_tablet_manager.cc:1875] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.188019 31291 ts_tablet_manager.cc:1888] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.188089 31291 log.cc:1196] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/c13e8de8518a4965a82bf46edcadd996
I20240502 01:58:50.188354 31291 ts_tablet_manager.cc:1909] T c13e8de8518a4965a82bf46edcadd996 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:50.192541 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet c13e8de8518a4965a82bf46edcadd996 (table rebalancer_dynamic_table_new_name [id=d6e0c8ba2359452cafca16293bd2a10b]) successfully deleted
I20240502 01:58:50.198873 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.232901 26794 consensus_queue.cc:237] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.234005 29213 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.234125 26930 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.234414 30910 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.234581 30608 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.236148 30924 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.236527 26930 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.237385 29213 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.238893 26506 catalog_manager.cc:5484] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.241611 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet b265eec55ee1406b9ff99c159304fa65 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:50.241827 29704 tablet_replica.cc:330] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:50.241940 29704 raft_consensus.cc:2229] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.242089 29704 raft_consensus.cc:2258] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.242590 29704 ts_tablet_manager.cc:1875] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:50.244642 29704 ts_tablet_manager.cc:1888] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:50.244726 29704 log.cc:1196] T b265eec55ee1406b9ff99c159304fa65 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/b265eec55ee1406b9ff99c159304fa65
I20240502 01:58:50.245218 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet b265eec55ee1406b9ff99c159304fa65 (table rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]) successfully deleted
I20240502 01:58:50.259588 29992 consensus_queue.cc:1035] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.262514 31155 consensus_queue.cc:1035] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.266052 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:50.272195 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 7ccdf5370b344060a0732e40d9cda34a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19]), partition=
I20240502 01:58:50.272207 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 7ccdf5370b344060a0732e40d9cda34a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19]), partition=
I20240502 01:58:50.272243 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 7ccdf5370b344060a0732e40d9cda34a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19]), partition=
I20240502 01:58:50.272436 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ccdf5370b344060a0732e40d9cda34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.272444 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ccdf5370b344060a0732e40d9cda34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.272464 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ccdf5370b344060a0732e40d9cda34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.274068 31044 tablet_bootstrap.cc:492] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:50.274441 31045 tablet_bootstrap.cc:492] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:50.274638 31132 tablet_bootstrap.cc:492] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:50.274999 31044 tablet_bootstrap.cc:654] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.275256 31045 tablet_bootstrap.cc:654] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.275660 31132 tablet_bootstrap.cc:654] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.276408 31045 tablet_bootstrap.cc:492] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:50.276468 31044 tablet_bootstrap.cc:492] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:50.276484 31045 ts_tablet_manager.cc:1367] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:50.276531 31044 ts_tablet_manager.cc:1367] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:50.276908 31045 raft_consensus.cc:348] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.276926 31044 raft_consensus.cc:348] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.277001 31045 raft_consensus.cc:374] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.277007 31044 raft_consensus.cc:374] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.277031 31045 raft_consensus.cc:729] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.277037 31044 raft_consensus.cc:729] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.277096 31132 tablet_bootstrap.cc:492] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:50.277165 31132 ts_tablet_manager.cc:1367] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:50.277139 31044 consensus_queue.cc:260] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.277139 31045 consensus_queue.cc:260] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.277396 31044 ts_tablet_manager.cc:1398] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.277396 31045 ts_tablet_manager.cc:1398] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:50.277613 31132 raft_consensus.cc:348] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.277732 31132 raft_consensus.cc:374] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.277761 31132 raft_consensus.cc:729] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.277870 31132 consensus_queue.cc:260] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.278143 31132 ts_tablet_manager.cc:1398] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.309362 31066 consensus_queue.cc:1035] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.315749 31066 consensus_queue.cc:1035] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.378909 29563 raft_consensus.cc:482] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:50.379010 29563 raft_consensus.cc:504] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.379338 29563 leader_election.cc:288] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:50.379658 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccdf5370b344060a0732e40d9cda34a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:50.379796 29213 raft_consensus.cc:2454] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:50.379832 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccdf5370b344060a0732e40d9cda34a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:50.379947 29488 raft_consensus.cc:2454] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:50.380072 29015 leader_election.cc:302] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:50.380316 29563 raft_consensus.cc:2781] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:50.380373 29563 raft_consensus.cc:482] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:50.380406 29563 raft_consensus.cc:3037] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.381338 29563 raft_consensus.cc:504] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.381659 29563 leader_election.cc:288] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:50.381940 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccdf5370b344060a0732e40d9cda34a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:50.382076 29488 raft_consensus.cc:3037] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.382185 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ccdf5370b344060a0732e40d9cda34a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:50.382275 29213 raft_consensus.cc:3037] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.383011 29488 raft_consensus.cc:2454] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:50.383114 29213 raft_consensus.cc:2454] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:50.383298 29016 leader_election.cc:302] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:50.383469 29563 raft_consensus.cc:2781] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:50.383553 29563 raft_consensus.cc:686] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:50.383679 29563 consensus_queue.cc:237] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.386509 26511 catalog_manager.cc:5484] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.397045 30909 consensus_queue.cc:1035] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.399824 29563 consensus_queue.cc:1035] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.411409 29196 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:50.414080 27042 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.415323 29330 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.420080 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.428890 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:50.430796 29451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.434885 26907 tablet_service.cc:1467] Processing CreateTablet for tablet bde4108189cc45a5a1e8ae2d0ef699fd (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.435159 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde4108189cc45a5a1e8ae2d0ef699fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.435698 29451 tablet_service.cc:1467] Processing CreateTablet for tablet c455f182f874431a88a46a8aff2b1675 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.435920 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c455f182f874431a88a46a8aff2b1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.436689 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 0831dc2504214ec69f0e5d9556fe6f1e (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.436908 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0831dc2504214ec69f0e5d9556fe6f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.437484 31378 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:50.437867 31044 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:50.438336 31378 tablet_bootstrap.cc:654] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.438688 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 0831dc2504214ec69f0e5d9556fe6f1e (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.438752 31044 tablet_bootstrap.cc:654] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.438951 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0831dc2504214ec69f0e5d9556fe6f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.439205 29330 tablet_service.cc:1467] Processing CreateTablet for tablet bde4108189cc45a5a1e8ae2d0ef699fd (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.439443 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:50.439468 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde4108189cc45a5a1e8ae2d0ef699fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.439478 29196 tablet_service.cc:1467] Processing CreateTablet for tablet c455f182f874431a88a46a8aff2b1675 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.439673 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c455f182f874431a88a46a8aff2b1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.440935 31045 tablet_bootstrap.cc:492] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:50.441653 26907 tablet_service.cc:1467] Processing CreateTablet for tablet c455f182f874431a88a46a8aff2b1675 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.441663 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 0831dc2504214ec69f0e5d9556fe6f1e (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.441794 31045 tablet_bootstrap.cc:654] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.441859 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c455f182f874431a88a46a8aff2b1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.441865 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0831dc2504214ec69f0e5d9556fe6f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.443929 26774 tablet_service.cc:1467] Processing CreateTablet for tablet bde4108189cc45a5a1e8ae2d0ef699fd (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:50.444175 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde4108189cc45a5a1e8ae2d0ef699fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.454125 31378 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:50.454206 31378 ts_tablet_manager.cc:1367] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:50.454602 31378 raft_consensus.cc:348] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.454689 31378 raft_consensus.cc:374] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.454711 31378 raft_consensus.cc:729] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.454823 31378 consensus_queue.cc:260] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.455077 31378 ts_tablet_manager.cc:1398] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.455219 31378 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:50.455994 31378 tablet_bootstrap.cc:654] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.456569 26907 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:50.457136 31378 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:50.457207 31378 ts_tablet_manager.cc:1367] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:50.457589 31378 raft_consensus.cc:348] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.457674 31378 raft_consensus.cc:374] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.457697 31378 raft_consensus.cc:729] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.457795 31378 consensus_queue.cc:260] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.458039 31378 ts_tablet_manager.cc:1398] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.461113 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:50.463572 31382 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:50.463594 31044 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:50.463670 31044 ts_tablet_manager.cc:1367] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 01:58:50.464089 31044 raft_consensus.cc:348] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.464212 31044 raft_consensus.cc:374] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.464243 31044 raft_consensus.cc:729] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.464357 31044 consensus_queue.cc:260] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.464607 31382 tablet_bootstrap.cc:654] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.464610 31044 ts_tablet_manager.cc:1398] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.464766 31044 tablet_bootstrap.cc:492] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:50.465464 31380 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:50.465658 31044 tablet_bootstrap.cc:654] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.466377 31380 tablet_bootstrap.cc:654] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.467852 31382 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:50.467953 31382 ts_tablet_manager.cc:1367] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:50.468324 31382 raft_consensus.cc:348] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.468425 31382 raft_consensus.cc:374] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.468454 31382 raft_consensus.cc:729] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.468557 31382 consensus_queue.cc:260] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.468842 31382 ts_tablet_manager.cc:1398] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.469156 31045 tablet_bootstrap.cc:492] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:50.469249 31045 ts_tablet_manager.cc:1367] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.001s
I20240502 01:58:50.469514 31380 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:50.469596 31380 ts_tablet_manager.cc:1367] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:50.469722 31045 raft_consensus.cc:348] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.469828 31045 raft_consensus.cc:374] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.469856 31045 raft_consensus.cc:729] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.469956 31045 consensus_queue.cc:260] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.470050 31380 raft_consensus.cc:348] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.470161 31380 raft_consensus.cc:374] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.470197 31380 raft_consensus.cc:729] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.470238 31045 ts_tablet_manager.cc:1398] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.470314 31380 consensus_queue.cc:260] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.470414 31045 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:50.470575 31380 ts_tablet_manager.cc:1398] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.470711 31380 tablet_bootstrap.cc:492] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:50.471506 31380 tablet_bootstrap.cc:654] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.472635 31380 tablet_bootstrap.cc:492] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:50.472707 31380 ts_tablet_manager.cc:1367] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:50.472915 31045 tablet_bootstrap.cc:654] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.473080 31380 raft_consensus.cc:348] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.473166 31380 raft_consensus.cc:374] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.473194 31380 raft_consensus.cc:729] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.473290 31380 consensus_queue.cc:260] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.473549 31380 ts_tablet_manager.cc:1398] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:50.479137 26513 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 01:58:50.479254 26513 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 01:58:50.481549 31044 tablet_bootstrap.cc:492] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:50.481557 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet 507cc922d2884f48b0f4c43185b4225d
I20240502 01:58:50.481624 31044 ts_tablet_manager.cc:1367] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:50.482090 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 507cc922d2884f48b0f4c43185b4225d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:50.482093 31044 raft_consensus.cc:348] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.482148 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 507cc922d2884f48b0f4c43185b4225d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:50.482189 31044 raft_consensus.cc:374] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.482219 31044 raft_consensus.cc:729] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.482249 30496 tablet_replica.cc:330] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:50.482223 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 507cc922d2884f48b0f4c43185b4225d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:50.482308 29704 tablet_replica.cc:330] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:50.482326 31044 consensus_queue.cc:260] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.482372 30496 raft_consensus.cc:2229] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:50.482398 29703 tablet_replica.cc:330] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:50.482421 29704 raft_consensus.cc:2229] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.482527 29703 raft_consensus.cc:2229] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.482559 30496 raft_consensus.cc:2258] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.482573 29704 raft_consensus.cc:2258] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.482589 31044 ts_tablet_manager.cc:1398] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.482664 29703 raft_consensus.cc:2258] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.483039 30496 ts_tablet_manager.cc:1875] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.483042 29704 ts_tablet_manager.cc:1875] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.483150 29703 ts_tablet_manager.cc:1875] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.485114 29704 ts_tablet_manager.cc:1888] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.485113 30496 ts_tablet_manager.cc:1888] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.485175 29704 log.cc:1196] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/507cc922d2884f48b0f4c43185b4225d
I20240502 01:58:50.485195 30496 log.cc:1196] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/507cc922d2884f48b0f4c43185b4225d
I20240502 01:58:50.485183 29703 ts_tablet_manager.cc:1888] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.485241 29703 log.cc:1196] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/507cc922d2884f48b0f4c43185b4225d
I20240502 01:58:50.485424 29704 ts_tablet_manager.cc:1909] T 507cc922d2884f48b0f4c43185b4225d P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:50.485472 29703 ts_tablet_manager.cc:1909] T 507cc922d2884f48b0f4c43185b4225d P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:50.485828 30496 ts_tablet_manager.cc:1909] T 507cc922d2884f48b0f4c43185b4225d P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:50.486264 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 507cc922d2884f48b0f4c43185b4225d (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.486317 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 507cc922d2884f48b0f4c43185b4225d (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.486644 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 507cc922d2884f48b0f4c43185b4225d (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.487251 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 507cc922d2884f48b0f4c43185b4225d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:50.487589 31390 tablet_replica.cc:330] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:50.487746 31390 raft_consensus.cc:2229] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.487895 31390 raft_consensus.cc:2258] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.488467 31390 ts_tablet_manager.cc:1875] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.487251 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 6e2ebe00a52240cb9ac5fc8fea9d7151
I20240502 01:58:50.489871 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:50.490011 30598 tablet_replica.cc:330] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:50.490130 30598 raft_consensus.cc:2229] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.490131 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 7efa5c32e6914d818262d473d71599a8
I20240502 01:58:50.490187 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:50.490249 30598 raft_consensus.cc:2258] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.490322 30496 tablet_replica.cc:330] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:50.490420 30496 raft_consensus.cc:2229] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:50.490655 30598 ts_tablet_manager.cc:1875] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.490746 30496 raft_consensus.cc:2258] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.490912 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efa5c32e6914d818262d473d71599a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:50.491166 30496 ts_tablet_manager.cc:1875] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.491863 29704 tablet_replica.cc:330] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:50.491981 29704 raft_consensus.cc:2229] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.492105 29704 raft_consensus.cc:2258] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.492540 29704 ts_tablet_manager.cc:1875] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.492580 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efa5c32e6914d818262d473d71599a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:50.492645 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:50.492717 29703 tablet_replica.cc:330] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:50.492779 31291 tablet_replica.cc:330] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:50.492825 29703 raft_consensus.cc:2229] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.492884 31291 raft_consensus.cc:2229] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.492928 29703 raft_consensus.cc:2258] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.492960 30496 ts_tablet_manager.cc:1888] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:50.492976 31291 raft_consensus.cc:2258] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.493016 30496 log.cc:1196] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/6e2ebe00a52240cb9ac5fc8fea9d7151
I20240502 01:58:50.493232 30496 ts_tablet_manager.cc:1909] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:50.493358 29703 ts_tablet_manager.cc:1875] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.493389 31291 ts_tablet_manager.cc:1875] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.494144 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.494520 31045 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:50.494593 31045 ts_tablet_manager.cc:1367] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
I20240502 01:58:50.495031 31045 raft_consensus.cc:348] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.495141 31045 raft_consensus.cc:374] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.495136 29703 ts_tablet_manager.cc:1888] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:50.495169 31045 raft_consensus.cc:729] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.495199 29703 log.cc:1196] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/7efa5c32e6914d818262d473d71599a8
I20240502 01:58:50.495271 31291 ts_tablet_manager.cc:1888] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:50.495270 31045 consensus_queue.cc:260] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.495323 31291 log.cc:1196] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/6e2ebe00a52240cb9ac5fc8fea9d7151
I20240502 01:58:50.495433 29703 ts_tablet_manager.cc:1909] T 7efa5c32e6914d818262d473d71599a8 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:50.495543 31291 ts_tablet_manager.cc:1909] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:50.496068 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.496325 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efa5c32e6914d818262d473d71599a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:50.496547 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 7efa5c32e6914d818262d473d71599a8 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.496711 30598 ts_tablet_manager.cc:1888] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:50.496759 30598 log.cc:1196] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/6e2ebe00a52240cb9ac5fc8fea9d7151
I20240502 01:58:50.496977 30598 ts_tablet_manager.cc:1909] T 6e2ebe00a52240cb9ac5fc8fea9d7151 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:50.497416 30598 tablet_replica.cc:330] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:50.497520 30598 raft_consensus.cc:2229] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:50.497483 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 6e2ebe00a52240cb9ac5fc8fea9d7151 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.497676 30598 raft_consensus.cc:2258] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.498503 30598 ts_tablet_manager.cc:1875] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.498570 29704 ts_tablet_manager.cc:1888] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:50.498644 29704 log.cc:1196] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/7efa5c32e6914d818262d473d71599a8
I20240502 01:58:50.498911 29704 ts_tablet_manager.cc:1909] T 7efa5c32e6914d818262d473d71599a8 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:50.499614 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 7efa5c32e6914d818262d473d71599a8 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.499596 29726 raft_consensus.cc:482] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:50.499692 29726 raft_consensus.cc:504] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.499759 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:50.500084 29726 leader_election.cc:288] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:50.500394 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831dc2504214ec69f0e5d9556fe6f1e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:50.500494 30598 ts_tablet_manager.cc:1888] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:50.500512 29349 raft_consensus.cc:2454] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:50.500630 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831dc2504214ec69f0e5d9556fe6f1e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:50.500737 29488 raft_consensus.cc:2454] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:50.500794 29147 leader_election.cc:302] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:50.500896 30598 log.cc:1196] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/7efa5c32e6914d818262d473d71599a8
I20240502 01:58:50.501122 30598 ts_tablet_manager.cc:1909] T 7efa5c32e6914d818262d473d71599a8 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:50.501600 29726 raft_consensus.cc:2781] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:50.501657 29726 raft_consensus.cc:482] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:50.501691 29726 raft_consensus.cc:3037] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.501718 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 7efa5c32e6914d818262d473d71599a8 (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.502961 29726 raft_consensus.cc:504] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.503268 29726 leader_election.cc:288] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:50.503566 31390 ts_tablet_manager.cc:1888] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.503626 31390 log.cc:1196] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/507cc922d2884f48b0f4c43185b4225d
I20240502 01:58:50.503988 31390 ts_tablet_manager.cc:1909] T 507cc922d2884f48b0f4c43185b4225d P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:50.504041 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831dc2504214ec69f0e5d9556fe6f1e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:50.504140 29488 raft_consensus.cc:3037] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.504329 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831dc2504214ec69f0e5d9556fe6f1e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:50.504434 29349 raft_consensus.cc:3037] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.504631 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 507cc922d2884f48b0f4c43185b4225d (table rebalance_test_table_16 [id=74ba0b4ded494b7da3c1f15a957044ad]) successfully deleted
I20240502 01:58:50.504988 29488 raft_consensus.cc:2454] T 0831dc2504214ec69f0e5d9556fe6f1e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:50.505277 29349 raft_consensus.cc:2454] T 0831dc2504214ec69f0e5d9556fe6f1e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:50.505609 29147 leader_election.cc:302] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:50.505771 29726 raft_consensus.cc:2781] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:50.505841 29726 raft_consensus.cc:686] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:50.506081 29488 raft_consensus.cc:1261] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:50.506104 29213 raft_consensus.cc:1261] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:50.505954 29726 consensus_queue.cc:237] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.506472 30909 consensus_queue.cc:1035] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.507333 31045 ts_tablet_manager.cc:1398] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20240502 01:58:50.507624 30909 consensus_queue.cc:1035] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.509116 31152 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:50.509398 31152 tablet.cc:1620] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:50.509338 26513 catalog_manager.cc:5484] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:50.509536 26513 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 7ccdf5370b344060a0732e40d9cda34a (table rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19]). Expected version 1 got 0
I20240502 01:58:50.511003 26511 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 7ccdf5370b344060a0732e40d9cda34a (table rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19]). Expected version 1 got 0
I20240502 01:58:50.514482 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19] alter complete (version 1)
I20240502 01:58:50.518299 31395 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:50.518558 31395 tablet.cc:1620] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:50.520802 30871 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:50.521054 30871 tablet.cc:1620] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:50.521775 26513 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 7ccdf5370b344060a0732e40d9cda34a (table rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19]). Expected version 1 got 0
I20240502 01:58:50.534852 31209 raft_consensus.cc:482] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:50.534950 31209 raft_consensus.cc:504] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.535320 31209 leader_election.cc:288] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:50.535619 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c455f182f874431a88a46a8aff2b1675" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:50.535622 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c455f182f874431a88a46a8aff2b1675" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:50.535737 29488 raft_consensus.cc:2454] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:50.535737 29213 raft_consensus.cc:2454] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:50.535984 26861 leader_election.cc:302] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:50.536132 31209 raft_consensus.cc:2781] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:50.536180 31209 raft_consensus.cc:482] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:50.536221 31209 raft_consensus.cc:3037] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.537184 31209 raft_consensus.cc:504] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.537525 31209 leader_election.cc:288] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:50.537776 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c455f182f874431a88a46a8aff2b1675" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:50.537777 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c455f182f874431a88a46a8aff2b1675" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:50.537882 29213 raft_consensus.cc:3037] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.537882 29488 raft_consensus.cc:3037] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.538769 29213 raft_consensus.cc:2454] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:50.538750 29488 raft_consensus.cc:2454] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:50.539094 26861 leader_election.cc:302] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:50.539291 31209 raft_consensus.cc:2781] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:50.539379 31209 raft_consensus.cc:686] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:50.539508 31209 consensus_queue.cc:237] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:50.541831 26506 catalog_manager.cc:5484] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:50.542693 26794 consensus_queue.cc:237] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:50.543929 29213 raft_consensus.cc:1261] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.544059 26661 raft_consensus.cc:1261] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.544322 30910 consensus_queue.cc:1035] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.544785 30910 consensus_queue.cc:1035] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.545943 30924 raft_consensus.cc:2932] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.546376 29213 raft_consensus.cc:2932] T cdd32c8294b44540ac5b832cc48d2519 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.547039 26661 raft_consensus.cc:2932] T cdd32c8294b44540ac5b832cc48d2519 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.549248 26513 catalog_manager.cc:5484] T cdd32c8294b44540ac5b832cc48d2519 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:50.551498 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet cdd32c8294b44540ac5b832cc48d2519 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:50.551703 29704 tablet_replica.cc:330] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:50.551820 29704 raft_consensus.cc:2229] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.551931 29704 raft_consensus.cc:2258] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.552320 29704 ts_tablet_manager.cc:1875] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:50.553985 29704 ts_tablet_manager.cc:1888] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:50.554064 29704 log.cc:1196] T cdd32c8294b44540ac5b832cc48d2519 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/cdd32c8294b44540ac5b832cc48d2519
I20240502 01:58:50.554454 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet cdd32c8294b44540ac5b832cc48d2519 (table rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]) successfully deleted
I20240502 01:58:50.621342 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:50.626945 31395 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:50.627180 30910 raft_consensus.cc:482] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:50.627310 31395 tablet.cc:1620] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:50.627274 30910 raft_consensus.cc:504] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.627637 30910 leader_election.cc:288] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:50.628079 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde4108189cc45a5a1e8ae2d0ef699fd" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:50.628221 29349 raft_consensus.cc:2454] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:50.628330 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde4108189cc45a5a1e8ae2d0ef699fd" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:50.628463 26927 raft_consensus.cc:2454] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:50.628592 26727 leader_election.cc:302] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:50.628746 30910 raft_consensus.cc:2781] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:50.628818 30910 raft_consensus.cc:482] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:50.628857 30910 raft_consensus.cc:3037] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.629169 31152 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:50.629462 31152 tablet.cc:1620] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:50.629895 30910 raft_consensus.cc:504] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.631991 30871 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:50.632277 30871 tablet.cc:1620] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:50.634769 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=7ff8ff662a1f4a31be50ae210944ab19] alter complete (version 2)
I20240502 01:58:50.635038 30910 leader_election.cc:288] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:50.635478 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde4108189cc45a5a1e8ae2d0ef699fd" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:50.635599 26927 raft_consensus.cc:3037] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.636584 26927 raft_consensus.cc:2454] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:50.636829 26730 leader_election.cc:302] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:50.636976 30910 raft_consensus.cc:2781] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:50.637058 30910 raft_consensus.cc:686] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:50.637190 30910 consensus_queue.cc:237] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:50.637208 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde4108189cc45a5a1e8ae2d0ef699fd" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:50.637332 29349 raft_consensus.cc:3037] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:50.638298 29349 raft_consensus.cc:2454] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:50.638943 26510 catalog_manager.cc:5484] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:50.652367 29196 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:50.653990 27042 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.659170 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.666911 29329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.668756 26907 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:50.670253 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:50.685424 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:50.687625 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:50.727277 27064 consensus_queue.cc:237] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:50.728435 29080 raft_consensus.cc:1261] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.728531 26661 raft_consensus.cc:1261] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.728888 31362 consensus_queue.cc:1035] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.729069 28915 consensus_queue.cc:1035] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.730664 31361 raft_consensus.cc:2932] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:50.731105 26661 raft_consensus.cc:2932] T c187cf1abe124e73badcb06ca1a489c4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:50.731251 29080 raft_consensus.cc:2932] T c187cf1abe124e73badcb06ca1a489c4 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:50.733608 26511 catalog_manager.cc:5484] T c187cf1abe124e73badcb06ca1a489c4 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:50.736306 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet c187cf1abe124e73badcb06ca1a489c4 with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:50.736980 30496 tablet_replica.cc:330] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:50.737129 30496 raft_consensus.cc:2229] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.737265 30496 raft_consensus.cc:2258] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.737754 30496 ts_tablet_manager.cc:1875] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:50.738433 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:50.739900 30496 ts_tablet_manager.cc:1888] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:50.739990 30496 log.cc:1196] T c187cf1abe124e73badcb06ca1a489c4 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/c187cf1abe124e73badcb06ca1a489c4
I20240502 01:58:50.740448 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet c187cf1abe124e73badcb06ca1a489c4 (table rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]) successfully deleted
I20240502 01:58:50.743479 31395 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:50.743795 31152 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:50.744395 30871 tablet.cc:1722] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:50.748414 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=7ff8ff662a1f4a31be50ae210944ab19] alter complete (version 3)
I20240502 01:58:50.812820 29196 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:50.813715 29329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.825327 27042 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:50.829281 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:50.832509 26907 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:50.839558 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:50.852329 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:50.853403 26506 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:50.853528 26506 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:50.855719 26506 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 7ccdf5370b344060a0732e40d9cda34a
I20240502 01:58:50.856158 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ccdf5370b344060a0732e40d9cda34a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:50.856398 31291 tablet_replica.cc:330] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:50.856408 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ccdf5370b344060a0732e40d9cda34a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:50.856525 31291 raft_consensus.cc:2229] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.856580 31390 tablet_replica.cc:330] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:50.856565 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ccdf5370b344060a0732e40d9cda34a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:50.856634 31291 raft_consensus.cc:2258] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.856693 31390 raft_consensus.cc:2229] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:50.856861 31390 raft_consensus.cc:2258] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.856976 31537 tablet_replica.cc:330] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:50.857100 31537 raft_consensus.cc:2229] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.857218 31537 raft_consensus.cc:2258] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.857261 31291 ts_tablet_manager.cc:1875] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.857795 31537 ts_tablet_manager.cc:1875] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.859437 31291 ts_tablet_manager.cc:1888] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.859495 31291 log.cc:1196] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/7ccdf5370b344060a0732e40d9cda34a
I20240502 01:58:50.859575 31537 ts_tablet_manager.cc:1888] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.859640 31537 log.cc:1196] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/7ccdf5370b344060a0732e40d9cda34a
I20240502 01:58:50.859715 31291 ts_tablet_manager.cc:1909] T 7ccdf5370b344060a0732e40d9cda34a P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:50.859894 31537 ts_tablet_manager.cc:1909] T 7ccdf5370b344060a0732e40d9cda34a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:50.860507 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 7ccdf5370b344060a0732e40d9cda34a (table rebalancer_dynamic_table_new_name [id=7ff8ff662a1f4a31be50ae210944ab19]) successfully deleted
I20240502 01:58:50.860822 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 7ccdf5370b344060a0732e40d9cda34a (table rebalancer_dynamic_table_new_name [id=7ff8ff662a1f4a31be50ae210944ab19]) successfully deleted
I20240502 01:58:50.861660 31390 ts_tablet_manager.cc:1875] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.863315 31390 ts_tablet_manager.cc:1888] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:50.863384 31390 log.cc:1196] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/7ccdf5370b344060a0732e40d9cda34a
I20240502 01:58:50.863549 31390 ts_tablet_manager.cc:1909] T 7ccdf5370b344060a0732e40d9cda34a P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:50.865265 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 7ccdf5370b344060a0732e40d9cda34a (table rebalancer_dynamic_table_new_name [id=7ff8ff662a1f4a31be50ae210944ab19]) successfully deleted
I20240502 01:58:50.865366 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:50.902792 26930 consensus_queue.cc:237] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:50.903973 29349 raft_consensus.cc:1261] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.903988 26794 raft_consensus.cc:1261] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:50.904376 31209 consensus_queue.cc:1035] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.904611 31086 consensus_queue.cc:1035] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:50.906109 31209 raft_consensus.cc:2932] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:50.906423 29349 raft_consensus.cc:2932] T 039270108deb4f55b8458ceccf31daa6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:50.906829 26794 raft_consensus.cc:2932] T 039270108deb4f55b8458ceccf31daa6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:50.909193 26506 catalog_manager.cc:5484] T 039270108deb4f55b8458ceccf31daa6 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:50.911972 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 039270108deb4f55b8458ceccf31daa6 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:50.912170 29704 tablet_replica.cc:330] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:50.912309 29704 raft_consensus.cc:2229] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.912446 29704 raft_consensus.cc:2258] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.912956 29704 ts_tablet_manager.cc:1875] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:50.914801 29704 ts_tablet_manager.cc:1888] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:50.914851 29704 log.cc:1196] T 039270108deb4f55b8458ceccf31daa6 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/039270108deb4f55b8458ceccf31daa6
I20240502 01:58:50.915335 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 039270108deb4f55b8458ceccf31daa6 (table rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]) successfully deleted
I20240502 01:58:50.918170 31391 consensus_queue.cc:1035] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.921592 31391 consensus_queue.cc:1035] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.930935 31209 consensus_queue.cc:1035] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.932315 31209 consensus_queue.cc:1035] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:50.957192 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:50.965881 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 4222878c30694f2f8bd68f31c5f02c7e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]), partition=
I20240502 01:58:50.965893 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 4222878c30694f2f8bd68f31c5f02c7e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]), partition=
I20240502 01:58:50.965893 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 4222878c30694f2f8bd68f31c5f02c7e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]), partition=
I20240502 01:58:50.966140 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4222878c30694f2f8bd68f31c5f02c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.966167 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4222878c30694f2f8bd68f31c5f02c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.967837 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4222878c30694f2f8bd68f31c5f02c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:50.968312 31044 tablet_bootstrap.cc:492] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:50.969126 31044 tablet_bootstrap.cc:654] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.969969 31380 tablet_bootstrap.cc:492] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:50.971225 31380 tablet_bootstrap.cc:654] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.972719 31044 tablet_bootstrap.cc:492] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:50.972798 31044 ts_tablet_manager.cc:1367] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:50.973181 31044 raft_consensus.cc:348] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:50.973284 31044 raft_consensus.cc:374] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.973325 31044 raft_consensus.cc:729] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.973429 31044 consensus_queue.cc:260] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:50.973672 31044 ts_tablet_manager.cc:1398] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.973830 31380 tablet_bootstrap.cc:492] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:50.973912 31380 ts_tablet_manager.cc:1367] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:50.974349 31380 raft_consensus.cc:348] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:50.974472 31380 raft_consensus.cc:374] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.974507 31380 raft_consensus.cc:729] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.974612 31380 consensus_queue.cc:260] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:50.974911 31380 ts_tablet_manager.cc:1398] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:50.979041 31602 tablet_bootstrap.cc:492] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:50.979923 31602 tablet_bootstrap.cc:654] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:50.980904 31602 tablet_bootstrap.cc:492] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:50.980983 31602 ts_tablet_manager.cc:1367] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:50.981400 31602 raft_consensus.cc:348] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:50.981518 31602 raft_consensus.cc:374] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:50.981559 31602 raft_consensus.cc:729] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:50.981674 31602 consensus_queue.cc:260] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:50.981932 31602 ts_tablet_manager.cc:1398] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:50.992277 26510 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 01:58:50.992394 26510 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 01:58:50.995195 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet d3e2d17af0194d5d8c08ad8372b1e9c1
I20240502 01:58:50.995513 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 408439eead244f4a8db70600cdc3ce65
I20240502 01:58:50.995688 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:50.995823 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 4 replicas of tablet 6e9b0960a2a94c3fb90902c126ccd034
I20240502 01:58:50.995873 29703 tablet_replica.cc:330] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:50.995914 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:50.995999 29703 raft_consensus.cc:2229] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.996050 30496 tablet_replica.cc:330] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:50.996150 30496 raft_consensus.cc:2229] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:50.996161 29703 raft_consensus.cc:2258] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.996349 30496 raft_consensus.cc:2258] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.996817 26907 tablet_service.cc:1514] Processing DeleteTablet for tablet 408439eead244f4a8db70600cdc3ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:50.996876 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:50.997017 30598 tablet_replica.cc:330] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:50.997068 30496 ts_tablet_manager.cc:1875] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.997105 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:50.997140 30598 raft_consensus.cc:2229] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:50.997272 30598 raft_consensus.cc:2258] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:50.997740 30598 ts_tablet_manager.cc:1875] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:50.998492 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:50.999388 30496 ts_tablet_manager.cc:1888] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:50.999449 30496 log.cc:1196] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/d3e2d17af0194d5d8c08ad8372b1e9c1
I20240502 01:58:50.999701 30496 ts_tablet_manager.cc:1909] T d3e2d17af0194d5d8c08ad8372b1e9c1 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:50.999837 30598 ts_tablet_manager.cc:1888] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:50.999900 30598 log.cc:1196] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/408439eead244f4a8db70600cdc3ce65
I20240502 01:58:51.000131 30598 ts_tablet_manager.cc:1909] T 408439eead244f4a8db70600cdc3ce65 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:51.000262 30496 tablet_replica.cc:330] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:51.000370 30496 raft_consensus.cc:2229] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:51.000380 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.000547 30496 raft_consensus.cc:2258] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.000661 30598 tablet_replica.cc:330] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:51.000725 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 408439eead244f4a8db70600cdc3ce65 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.000778 30598 raft_consensus.cc:2229] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.000913 30598 raft_consensus.cc:2258] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.000946 30496 ts_tablet_manager.cc:1875] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.001374 30598 ts_tablet_manager.cc:1875] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.002110 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 408439eead244f4a8db70600cdc3ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:51.002278 29704 tablet_replica.cc:330] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:51.002307 31291 tablet_replica.cc:330] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:51.002408 29704 raft_consensus.cc:2229] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:51.002418 31291 raft_consensus.cc:2229] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.002511 31291 raft_consensus.cc:2258] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.002615 29704 raft_consensus.cc:2258] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.002938 29703 ts_tablet_manager.cc:1875] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.002950 31291 ts_tablet_manager.cc:1875] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.003072 29704 ts_tablet_manager.cc:1875] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.004597 31291 ts_tablet_manager.cc:1888] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.004648 31291 log.cc:1196] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/6e9b0960a2a94c3fb90902c126ccd034
I20240502 01:58:51.004911 31291 ts_tablet_manager.cc:1909] T 6e9b0960a2a94c3fb90902c126ccd034 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:51.005071 29704 ts_tablet_manager.cc:1888] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:51.005072 29703 ts_tablet_manager.cc:1888] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:51.005134 29704 log.cc:1196] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/408439eead244f4a8db70600cdc3ce65
I20240502 01:58:51.005134 29703 log.cc:1196] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/d3e2d17af0194d5d8c08ad8372b1e9c1
I20240502 01:58:51.005365 29704 ts_tablet_manager.cc:1909] T 408439eead244f4a8db70600cdc3ce65 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:58:51.005365 29703 ts_tablet_manager.cc:1909] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:51.005470 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 6e9b0960a2a94c3fb90902c126ccd034 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.005441 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e2d17af0194d5d8c08ad8372b1e9c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:51.005656 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9b0960a2a94c3fb90902c126ccd034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:51.005807 31537 tablet_replica.cc:330] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:51.005930 31537 raft_consensus.cc:2229] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.006068 31537 raft_consensus.cc:2258] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.006822 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 408439eead244f4a8db70600cdc3ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:50 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:51.007174 29703 tablet_replica.cc:330] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:51.007261 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.007352 29703 raft_consensus.cc:2229] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.007503 29703 raft_consensus.cc:2258] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.007966 29703 ts_tablet_manager.cc:1875] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.008340 31537 ts_tablet_manager.cc:1875] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.009505 29703 ts_tablet_manager.cc:1888] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.009549 29703 log.cc:1196] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/6e9b0960a2a94c3fb90902c126ccd034
I20240502 01:58:51.009699 29703 ts_tablet_manager.cc:1909] T 6e9b0960a2a94c3fb90902c126ccd034 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:51.009979 31537 ts_tablet_manager.cc:1888] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 01:58:51.010066 31537 log.cc:1196] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/d3e2d17af0194d5d8c08ad8372b1e9c1
I20240502 01:58:51.010300 31537 ts_tablet_manager.cc:1909] T d3e2d17af0194d5d8c08ad8372b1e9c1 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:51.010973 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 408439eead244f4a8db70600cdc3ce65 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.011219 30598 ts_tablet_manager.cc:1888] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.011407 30598 log.cc:1196] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/6e9b0960a2a94c3fb90902c126ccd034
I20240502 01:58:51.011428 29703 tablet_replica.cc:330] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:51.011425 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 6e9b0960a2a94c3fb90902c126ccd034 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.011538 29703 raft_consensus.cc:2229] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.011709 29703 raft_consensus.cc:2258] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.011832 30598 ts_tablet_manager.cc:1909] T 6e9b0960a2a94c3fb90902c126ccd034 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:51.012342 29703 ts_tablet_manager.cc:1875] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.011548 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet d3e2d17af0194d5d8c08ad8372b1e9c1 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.012460 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 6e9b0960a2a94c3fb90902c126ccd034 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.013573 30496 ts_tablet_manager.cc:1888] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.013644 30496 log.cc:1196] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/6e9b0960a2a94c3fb90902c126ccd034
I20240502 01:58:51.013867 30496 ts_tablet_manager.cc:1909] T 6e9b0960a2a94c3fb90902c126ccd034 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:51.014323 29703 ts_tablet_manager.cc:1888] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 01:58:51.015520 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 6e9b0960a2a94c3fb90902c126ccd034 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.015714 29703 log.cc:1196] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/408439eead244f4a8db70600cdc3ce65
I20240502 01:58:51.016007 29703 ts_tablet_manager.cc:1909] T 408439eead244f4a8db70600cdc3ce65 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:51.017020 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 408439eead244f4a8db70600cdc3ce65 (table rebalance_test_table_18 [id=d591439fcb3647eea2d27e598437f007]) successfully deleted
I20240502 01:58:51.020313 29196 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:51.021342 30924 consensus_queue.cc:1035] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.022511 29329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:51.029062 26909 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.029716 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:51.035157 29058 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:51.036259 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:51.049651 26774 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:51.055896 30910 consensus_queue.cc:1035] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.059456 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:51.099246 27064 consensus_queue.cc:237] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:51.100479 29080 raft_consensus.cc:1261] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:51.100611 26930 raft_consensus.cc:1261] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:51.100893 31361 consensus_queue.cc:1035] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:51.101087 28915 consensus_queue.cc:1035] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:51.102713 31362 raft_consensus.cc:2932] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:51.103210 29080 raft_consensus.cc:2932] T ae9f78db63b74beca267ff5f12a211ae P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:51.104274 26930 raft_consensus.cc:2932] T ae9f78db63b74beca267ff5f12a211ae P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:51.106062 26510 catalog_manager.cc:5484] T ae9f78db63b74beca267ff5f12a211ae P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:51.109463 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet ae9f78db63b74beca267ff5f12a211ae with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:51.109661 30496 tablet_replica.cc:330] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:51.109795 30496 raft_consensus.cc:2229] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.109931 30496 raft_consensus.cc:2258] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.110419 30496 ts_tablet_manager.cc:1875] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:51.112352 30496 ts_tablet_manager.cc:1888] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:51.112421 30496 log.cc:1196] T ae9f78db63b74beca267ff5f12a211ae P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/ae9f78db63b74beca267ff5f12a211ae
I20240502 01:58:51.112867 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet ae9f78db63b74beca267ff5f12a211ae (table rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]) successfully deleted
I20240502 01:58:51.154141 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:51.161584 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 78609c45dc0c4e1b838bb737ebf7e034 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.161646 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 398742b2cb3e41f1bc16fa37a5168809 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.161903 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78609c45dc0c4e1b838bb737ebf7e034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.161916 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 398742b2cb3e41f1bc16fa37a5168809. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.162076 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 398742b2cb3e41f1bc16fa37a5168809 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.162325 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 398742b2cb3e41f1bc16fa37a5168809. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.162271 27044 tablet_service.cc:1467] Processing CreateTablet for tablet dd9b09843c784939bebe5695799e7916 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.162532 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9b09843c784939bebe5695799e7916. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.163944 31380 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:51.164381 31602 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:51.164681 31673 tablet_bootstrap.cc:492] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:51.164860 31380 tablet_bootstrap.cc:654] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.164851 27042 tablet_service.cc:1467] Processing CreateTablet for tablet 78609c45dc0c4e1b838bb737ebf7e034 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.165063 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78609c45dc0c4e1b838bb737ebf7e034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.165254 31602 tablet_bootstrap.cc:654] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.165745 31673 tablet_bootstrap.cc:654] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.166004 31044 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:51.166298 29329 tablet_service.cc:1467] Processing CreateTablet for tablet dd9b09843c784939bebe5695799e7916 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.166492 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9b09843c784939bebe5695799e7916. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.166896 31044 tablet_bootstrap.cc:654] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.166955 31673 tablet_bootstrap.cc:492] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:51.167025 31673 ts_tablet_manager.cc:1367] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:51.167466 31673 raft_consensus.cc:348] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.167559 31673 raft_consensus.cc:374] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.167590 31673 raft_consensus.cc:729] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.167687 31673 consensus_queue.cc:260] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.167937 31673 ts_tablet_manager.cc:1398] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:51.168073 31673 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:51.168207 31380 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:51.168275 31380 ts_tablet_manager.cc:1367] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:51.168669 31380 raft_consensus.cc:348] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.168762 31380 raft_consensus.cc:374] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.168793 31380 raft_consensus.cc:729] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.168860 31673 tablet_bootstrap.cc:654] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.168897 31380 consensus_queue.cc:260] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.169162 31380 ts_tablet_manager.cc:1398] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.169323 31380 tablet_bootstrap.cc:492] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:51.169551 29058 tablet_service.cc:1467] Processing CreateTablet for tablet dd9b09843c784939bebe5695799e7916 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.169781 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9b09843c784939bebe5695799e7916. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.170506 31044 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:51.170598 31044 ts_tablet_manager.cc:1367] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 01:58:51.170994 31044 raft_consensus.cc:348] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.171084 31044 raft_consensus.cc:374] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.171135 31044 raft_consensus.cc:729] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.171167 31380 tablet_bootstrap.cc:654] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.171243 31044 consensus_queue.cc:260] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.171506 31044 ts_tablet_manager.cc:1398] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:51.172785 31602 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:51.172878 31602 ts_tablet_manager.cc:1367] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:51.172905 31380 tablet_bootstrap.cc:492] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:51.172974 31380 ts_tablet_manager.cc:1367] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:51.173269 31602 raft_consensus.cc:348] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.173364 31602 raft_consensus.cc:374] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.173393 31602 raft_consensus.cc:729] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.173390 31380 raft_consensus.cc:348] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.173478 31380 raft_consensus.cc:374] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.173509 31380 raft_consensus.cc:729] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.173493 31602 consensus_queue.cc:260] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.173614 31380 consensus_queue.cc:260] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.173755 31602 ts_tablet_manager.cc:1398] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.173862 31380 ts_tablet_manager.cc:1398] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.173941 31602 tablet_bootstrap.cc:492] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:51.174736 31602 tablet_bootstrap.cc:654] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.176129 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 78609c45dc0c4e1b838bb737ebf7e034 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.176396 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78609c45dc0c4e1b838bb737ebf7e034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.177317 31602 tablet_bootstrap.cc:492] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:51.177397 31602 ts_tablet_manager.cc:1367] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:51.177770 31602 raft_consensus.cc:348] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.177862 31602 raft_consensus.cc:374] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.177891 31602 raft_consensus.cc:729] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.177989 31602 consensus_queue.cc:260] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.178260 31602 ts_tablet_manager.cc:1398] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.178707 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 398742b2cb3e41f1bc16fa37a5168809 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:51.178920 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 398742b2cb3e41f1bc16fa37a5168809. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.179353 31681 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:51.180332 31681 tablet_bootstrap.cc:654] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.181349 31681 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:51.181428 31681 ts_tablet_manager.cc:1367] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:51.181802 31673 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:51.181875 31673 ts_tablet_manager.cc:1367] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240502 01:58:51.181846 31681 raft_consensus.cc:348] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.181938 31681 raft_consensus.cc:374] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.181968 31681 raft_consensus.cc:729] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.182103 31681 consensus_queue.cc:260] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.182307 31673 raft_consensus.cc:348] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.182399 31673 raft_consensus.cc:374] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.182428 31673 raft_consensus.cc:729] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.182543 31673 consensus_queue.cc:260] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.182580 31681 ts_tablet_manager.cc:1398] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.182739 31681 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:51.183530 31681 tablet_bootstrap.cc:654] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.185019 31673 ts_tablet_manager.cc:1398] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:51.187754 31681 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:51.187837 31681 ts_tablet_manager.cc:1367] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:51.188236 31681 raft_consensus.cc:348] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.188325 31681 raft_consensus.cc:374] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.188346 31681 raft_consensus.cc:729] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.188439 31681 consensus_queue.cc:260] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.188688 31681 ts_tablet_manager.cc:1398] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:51.212997 31620 raft_consensus.cc:482] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:51.213080 31620 raft_consensus.cc:504] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.213497 31620 leader_election.cc:288] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:51.213761 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78609c45dc0c4e1b838bb737ebf7e034" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:51.213796 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78609c45dc0c4e1b838bb737ebf7e034" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:51.213886 27064 raft_consensus.cc:2454] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:51.213887 29213 raft_consensus.cc:2454] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:51.214184 29286 leader_election.cc:302] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:58:51.214466 31620 raft_consensus.cc:2781] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:51.214515 31620 raft_consensus.cc:482] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:51.214550 31620 raft_consensus.cc:3037] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.215278 31620 raft_consensus.cc:504] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.215576 31620 leader_election.cc:288] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:51.215867 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78609c45dc0c4e1b838bb737ebf7e034" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:51.215866 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78609c45dc0c4e1b838bb737ebf7e034" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:51.215967 29213 raft_consensus.cc:3037] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.215967 27064 raft_consensus.cc:3037] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.216765 27064 raft_consensus.cc:2454] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:51.216765 29213 raft_consensus.cc:2454] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:51.217027 29285 leader_election.cc:302] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:51.217165 31620 raft_consensus.cc:2781] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:51.217247 31620 raft_consensus.cc:686] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:51.217375 31620 consensus_queue.cc:237] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.218843 26510 catalog_manager.cc:5484] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:51.244895 31032 raft_consensus.cc:482] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:51.244997 31032 raft_consensus.cc:504] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:51.245409 31032 leader_election.cc:288] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:51.246014 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4222878c30694f2f8bd68f31c5f02c7e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:51.246160 29349 raft_consensus.cc:2454] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:51.246408 29419 leader_election.cc:302] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:51.246562 31032 raft_consensus.cc:2781] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:51.246629 31032 raft_consensus.cc:482] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:51.246666 31032 raft_consensus.cc:3037] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.247611 31032 raft_consensus.cc:504] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:51.247885 31032 leader_election.cc:288] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:51.248193 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4222878c30694f2f8bd68f31c5f02c7e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:51.248296 29349 raft_consensus.cc:3037] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.249070 29349 raft_consensus.cc:2454] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:51.249286 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4222878c30694f2f8bd68f31c5f02c7e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:51.249318 29419 leader_election.cc:302] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:51.249409 29080 raft_consensus.cc:2454] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:51.249483 31032 raft_consensus.cc:2781] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:51.249640 31032 raft_consensus.cc:686] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:51.249642 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4222878c30694f2f8bd68f31c5f02c7e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:51.249735 29079 raft_consensus.cc:3037] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.249773 31032 consensus_queue.cc:237] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:51.250607 29079 raft_consensus.cc:2454] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:51.251331 26510 catalog_manager.cc:5484] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:51.281194 26774 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:51.282599 29058 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:51.286633 29195 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:51.286881 29454 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.294220 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:51.294853 29329 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.303390 26909 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.322278 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:51.379988 26930 consensus_queue.cc:237] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:51.381381 29213 raft_consensus.cc:1261] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:51.381443 29488 raft_consensus.cc:1261] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:58:51.381616 26861 consensus_peers.cc:485] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c455f182f874431a88a46a8aff2b1675. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:51.381810 31086 consensus_queue.cc:1035] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:51.382000 31209 consensus_queue.cc:1035] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:51.383574 31209 raft_consensus.cc:2932] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:51.383945 29488 raft_consensus.cc:2932] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:51.385524 29213 raft_consensus.cc:2932] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:51.386734 26510 catalog_manager.cc:5484] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:51.387928 30909 raft_consensus.cc:482] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:51.388020 30909 raft_consensus.cc:504] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.388375 30909 leader_election.cc:288] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:51.388726 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b09843c784939bebe5695799e7916" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:51.388847 27064 raft_consensus.cc:2454] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:51.389134 29016 leader_election.cc:302] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:51.389132 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:51.390318 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b09843c784939bebe5695799e7916" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:51.390448 29349 raft_consensus.cc:2454] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:51.390648 30909 raft_consensus.cc:2781] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:51.390702 30909 raft_consensus.cc:482] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:51.390734 30909 raft_consensus.cc:3037] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.391544 30909 raft_consensus.cc:504] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.391840 30909 leader_election.cc:288] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:51.392165 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b09843c784939bebe5695799e7916" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:51.392247 27064 raft_consensus.cc:3037] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.393178 27064 raft_consensus.cc:2454] T dd9b09843c784939bebe5695799e7916 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:51.393862 29350 raft_consensus.cc:1261] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:51.394127 29016 leader_election.cc:302] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:51.394299 31686 consensus_queue.cc:1035] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.394719 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b09843c784939bebe5695799e7916" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:51.394819 29349 raft_consensus.cc:3037] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.394910 29079 raft_consensus.cc:1261] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:51.395368 31686 consensus_queue.cc:1035] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.395610 29349 raft_consensus.cc:2454] T dd9b09843c784939bebe5695799e7916 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:51.397205 30909 raft_consensus.cc:2781] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:51.397342 30909 raft_consensus.cc:686] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:51.397440 26511 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]). Expected version 1 got 0
I20240502 01:58:51.397470 30909 consensus_queue.cc:237] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:51.399240 26513 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]). Expected version 1 got 0
I20240502 01:58:51.399360 26513 catalog_manager.cc:5484] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:51.401434 31725 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:51.401692 26511 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]). Expected version 1 got 0
I20240502 01:58:51.401942 31725 tablet.cc:1620] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:51.402865 31727 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:51.403115 26511 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490]). Expected version 1 got 0
I20240502 01:58:51.403851 31729 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:51.404143 31729 tablet.cc:1620] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:51.403863 31727 tablet.cc:1620] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:51.407850 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490] alter complete (version 1)
I20240502 01:58:51.419303 26794 consensus_queue.cc:237] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
W20240502 01:58:51.420804 26729 consensus_peers.cc:485] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc99f8bfb06c4bcda6111b218125826c. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:51.420822 27064 raft_consensus.cc:1261] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:51.420835 26661 raft_consensus.cc:1261] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:51.421226 30910 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:51.421427 30924 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:51.423202 30910 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:51.423605 27064 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:51.425488 26661 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:51.426347 26513 catalog_manager.cc:5484] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:51.431898 31391 raft_consensus.cc:482] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:51.432010 31391 raft_consensus.cc:504] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.432492 31391 leader_election.cc:288] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:51.433571 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "398742b2cb3e41f1bc16fa37a5168809" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:51.433708 29488 raft_consensus.cc:2454] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:51.434005 29150 leader_election.cc:302] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:51.434193 31391 raft_consensus.cc:2781] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:51.434248 31391 raft_consensus.cc:482] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:51.434298 31391 raft_consensus.cc:3037] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.435221 31391 raft_consensus.cc:504] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.435565 31391 leader_election.cc:288] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:51.435827 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "398742b2cb3e41f1bc16fa37a5168809" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:51.435933 29488 raft_consensus.cc:3037] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.436793 29488 raft_consensus.cc:2454] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:51.437361 29150 leader_election.cc:302] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:51.437511 31391 raft_consensus.cc:2781] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:51.437642 31391 raft_consensus.cc:686] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:51.437769 31391 consensus_queue.cc:237] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.437829 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "398742b2cb3e41f1bc16fa37a5168809" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:51.437942 29079 raft_consensus.cc:2454] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:51.438166 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "398742b2cb3e41f1bc16fa37a5168809" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:51.438253 29080 raft_consensus.cc:3037] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:51.438977 29080 raft_consensus.cc:2454] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:51.439348 26511 catalog_manager.cc:5484] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:51.449215 26661 consensus_queue.cc:237] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:51.450529 26794 raft_consensus.cc:1261] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:51.450560 26592 consensus_peers.cc:485] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 50bfc86904bc4347bd6beb7f16eb638d. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:51.450590 26930 raft_consensus.cc:1261] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:51.450857 31766 consensus_queue.cc:1035] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:51.451059 31765 consensus_queue.cc:1035] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:51.452641 31766 raft_consensus.cc:2932] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:51.453055 26930 raft_consensus.cc:2932] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:51.455583 26511 catalog_manager.cc:5484] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:51.456769 26794 raft_consensus.cc:2932] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:51.473752 27064 consensus_queue.cc:237] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:51.475206 26661 raft_consensus.cc:1261] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:51.475543 31361 consensus_queue.cc:1035] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:51.475656 26930 raft_consensus.cc:1261] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:51.475726 26998 consensus_peers.cc:485] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dfdd6d487b57479cb48a92bc770947ce. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:51.476014 31361 consensus_queue.cc:1035] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:51.477474 28915 raft_consensus.cc:2932] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:51.477892 26930 raft_consensus.cc:2932] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:51.479261 26661 raft_consensus.cc:2932] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:51.480250 26510 catalog_manager.cc:5484] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:51.513115 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:51.518885 31725 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:51.519192 31725 tablet.cc:1620] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:51.519210 31727 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:51.519482 31727 tablet.cc:1620] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:51.522540 31729 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:51.522821 31729 tablet.cc:1620] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:51.524462 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=fbfb5e4da94848e38da0799868b0d490] alter complete (version 2)
I20240502 01:58:51.557880 29329 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.558976 29195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:51.563052 26909 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.579661 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:51.582607 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:51.582865 29454 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.590217 26774 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:51.593957 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.627450 26508 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:51.633013 31725 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:51.633967 31729 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:51.635108 31727 tablet.cc:1722] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:51.640666 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=fbfb5e4da94848e38da0799868b0d490] alter complete (version 3)
I20240502 01:58:51.702728 29454 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.703258 29195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:51.709975 29329 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.710402 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.710901 26909 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.713423 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:51.713644 26774 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:51.730121 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:51.743927 31620 consensus_queue.cc:1035] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.748747 26511 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:51.748878 26511 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:51.750846 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 4222878c30694f2f8bd68f31c5f02c7e
I20240502 01:58:51.751389 29329 tablet_service.cc:1514] Processing DeleteTablet for tablet 4222878c30694f2f8bd68f31c5f02c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:51 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:51.751391 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 4222878c30694f2f8bd68f31c5f02c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:51 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:51.752030 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet 4222878c30694f2f8bd68f31c5f02c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:51 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:51.753729 31896 tablet_replica.cc:330] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:51.753894 31896 raft_consensus.cc:2229] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:51.754109 31620 consensus_queue.cc:1035] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.754273 31896 raft_consensus.cc:2258] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.754909 31896 ts_tablet_manager.cc:1875] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.757308 31893 tablet_replica.cc:330] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:51.757500 31893 raft_consensus.cc:2229] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.757656 31893 raft_consensus.cc:2258] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.758226 31893 ts_tablet_manager.cc:1875] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.758643 31896 ts_tablet_manager.cc:1888] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.758765 31896 log.cc:1196] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/4222878c30694f2f8bd68f31c5f02c7e
I20240502 01:58:51.759107 31896 ts_tablet_manager.cc:1909] T 4222878c30694f2f8bd68f31c5f02c7e P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:51.759756 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table_new_name [id=fbfb5e4da94848e38da0799868b0d490]) successfully deleted
I20240502 01:58:51.760576 31893 ts_tablet_manager.cc:1888] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.760746 31893 log.cc:1196] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/4222878c30694f2f8bd68f31c5f02c7e
I20240502 01:58:51.761045 31893 ts_tablet_manager.cc:1909] T 4222878c30694f2f8bd68f31c5f02c7e P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:51.761607 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table_new_name [id=fbfb5e4da94848e38da0799868b0d490]) successfully deleted
I20240502 01:58:51.764011 31894 tablet_replica.cc:330] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:51.764173 31894 raft_consensus.cc:2229] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:51.764331 31894 raft_consensus.cc:2258] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:51.764946 31894 ts_tablet_manager.cc:1875] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:51.767122 31894 ts_tablet_manager.cc:1888] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:51.767201 31894 log.cc:1196] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/4222878c30694f2f8bd68f31c5f02c7e
I20240502 01:58:51.767400 31894 ts_tablet_manager.cc:1909] T 4222878c30694f2f8bd68f31c5f02c7e P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:51.767933 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 4222878c30694f2f8bd68f31c5f02c7e (table rebalancer_dynamic_table_new_name [id=fbfb5e4da94848e38da0799868b0d490]) successfully deleted
I20240502 01:58:51.853659 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:51.860754 29058 tablet_service.cc:1467] Processing CreateTablet for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]), partition=
I20240502 01:58:51.860821 26774 tablet_service.cc:1467] Processing CreateTablet for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]), partition=
I20240502 01:58:51.860905 29454 tablet_service.cc:1467] Processing CreateTablet for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]), partition=
I20240502 01:58:51.861037 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc35b3c50ed94ccfa748943f2cfeacf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.861079 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc35b3c50ed94ccfa748943f2cfeacf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.861119 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc35b3c50ed94ccfa748943f2cfeacf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.862957 31950 ts_tablet_manager.cc:902] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:51.864863 31953 tablet_bootstrap.cc:492] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:51.865754 31953 tablet_bootstrap.cc:654] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.869567 31950 tablet_copy_client.cc:308] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:51.870208 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet c455f182f874431a88a46a8aff2b1675 from peer 998b57e2badc4d779c586c6579149114 ({username='slave'} at 127.19.43.193:40669)
I20240502 01:58:51.870314 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet c455f182f874431a88a46a8aff2b1675 from peer 998b57e2badc4d779c586c6579149114 at {username='slave'} at 127.19.43.193:40669: session id = 998b57e2badc4d779c586c6579149114-c455f182f874431a88a46a8aff2b1675
I20240502 01:58:51.871233 26950 tablet_copy_source_session.cc:205] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:51.874536 31950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c455f182f874431a88a46a8aff2b1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.876065 31951 tablet_bootstrap.cc:492] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:51.876977 31951 tablet_bootstrap.cc:654] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.877740 31952 tablet_bootstrap.cc:492] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:51.878625 31952 tablet_bootstrap.cc:654] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:51.879717 31953 tablet_bootstrap.cc:492] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:51.879820 31953 ts_tablet_manager.cc:1367] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 01:58:51.880194 31950 tablet_copy_client.cc:783] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:51.880220 31953 raft_consensus.cc:348] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:51.880326 31953 raft_consensus.cc:374] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.880357 31953 raft_consensus.cc:729] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.880363 31950 tablet_copy_client.cc:647] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:51.880477 31953 consensus_queue.cc:260] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:51.880731 31953 ts_tablet_manager.cc:1398] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:51.880965 31961 ts_tablet_manager.cc:902] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:51.881723 31950 tablet_copy_client.cc:523] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:51.884028 31950 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:51.884630 31951 tablet_bootstrap.cc:492] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:51.884778 31951 ts_tablet_manager.cc:1367] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:51.886520 31951 raft_consensus.cc:348] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:51.886618 31961 tablet_copy_client.cc:308] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:51.886703 31951 raft_consensus.cc:374] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.886785 31951 raft_consensus.cc:729] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.886971 31951 consensus_queue.cc:260] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:51.887053 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet dfdd6d487b57479cb48a92bc770947ce from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:53349)
I20240502 01:58:51.887136 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet dfdd6d487b57479cb48a92bc770947ce from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:53349: session id = 7c693d0933c64d9babc07f404df45cd1-dfdd6d487b57479cb48a92bc770947ce
I20240502 01:58:51.888105 27083 tablet_copy_source_session.cc:205] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:51.889835 31961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdd6d487b57479cb48a92bc770947ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.889894 29195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:51.893893 31961 tablet_copy_client.cc:783] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:51.894569 31961 tablet_copy_client.cc:647] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:51.895129 31951 ts_tablet_manager.cc:1398] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240502 01:58:51.895709 31961 tablet_copy_client.cc:523] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:51.896781 29329 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:51.898293 31950 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:51.898738 31950 tablet_bootstrap.cc:492] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: Bootstrap complete.
I20240502 01:58:51.899219 31950 ts_tablet_manager.cc:1367] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20240502 01:58:51.899669 31950 raft_consensus.cc:348] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:51.899782 31950 raft_consensus.cc:729] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.899901 31950 consensus_queue.cc:260] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:51.901386 31961 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:51.902575 31950 ts_tablet_manager.cc:1398] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:51.903913 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 998b57e2badc4d779c586c6579149114-c455f182f874431a88a46a8aff2b1675 received from {username='slave'} at 127.19.43.193:40669
I20240502 01:58:51.903983 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 998b57e2badc4d779c586c6579149114-c455f182f874431a88a46a8aff2b1675 on tablet c455f182f874431a88a46a8aff2b1675 with peer 998b57e2badc4d779c586c6579149114
I20240502 01:58:51.906561 31974 ts_tablet_manager.cc:902] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:51.906994 31974 tablet_copy_client.cc:308] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:51.909456 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet dc99f8bfb06c4bcda6111b218125826c from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:58:51.909539 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet dc99f8bfb06c4bcda6111b218125826c from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-dc99f8bfb06c4bcda6111b218125826c
I20240502 01:58:51.910446 26815 tablet_copy_source_session.cc:205] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:51.914744 31974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc99f8bfb06c4bcda6111b218125826c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.917658 31974 tablet_copy_client.cc:783] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:51.917850 31974 tablet_copy_client.cc:647] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:51.921599 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:51.926878 31391 consensus_queue.cc:1035] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.931991 31961 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:51.932140 31974 tablet_copy_client.cc:523] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:51.932166 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:51.932852 31985 ts_tablet_manager.cc:902] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:51.933380 31974 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:51.933413 31985 tablet_copy_client.cc:308] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:58:51.933892 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet 50bfc86904bc4347bd6beb7f16eb638d from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:38479)
I20240502 01:58:51.933975 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet 50bfc86904bc4347bd6beb7f16eb638d from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:38479: session id = 468dd27456c64d67b22e8a58af93d93d-50bfc86904bc4347bd6beb7f16eb638d
I20240502 01:58:51.934262 31757 consensus_queue.cc:1035] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.934911 26682 tablet_copy_source_session.cc:205] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:51.935770 31985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bfc86904bc4347bd6beb7f16eb638d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:51.936111 26774 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:51.936836 31961 tablet_bootstrap.cc:492] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:51.937009 31961 ts_tablet_manager.cc:1367] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.036s user 0.021s sys 0.003s
I20240502 01:58:51.937399 31961 raft_consensus.cc:348] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:51.937515 31961 raft_consensus.cc:729] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.937638 31961 consensus_queue.cc:260] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:51.937999 31961 ts_tablet_manager.cc:1398] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.938431 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-dfdd6d487b57479cb48a92bc770947ce received from {username='slave'} at 127.19.43.200:53349
I20240502 01:58:51.938505 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-dfdd6d487b57479cb48a92bc770947ce on tablet dfdd6d487b57479cb48a92bc770947ce with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:51.938530 31985 tablet_copy_client.cc:783] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:51.938663 31985 tablet_copy_client.cc:647] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:51.941339 31985 tablet_copy_client.cc:523] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:51.942528 31985 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:51.944239 31952 tablet_bootstrap.cc:492] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:51.944324 31952 ts_tablet_manager.cc:1367] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.000s
I20240502 01:58:51.944732 31952 raft_consensus.cc:348] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:51.944829 31952 raft_consensus.cc:374] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:51.944856 31952 raft_consensus.cc:729] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.944957 31952 consensus_queue.cc:260] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:51.945195 31952 ts_tablet_manager.cc:1398] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:51.947436 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:51.953814 26909 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.959530 31724 consensus_queue.cc:1035] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:51.978816 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:51.991966 31974 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:51.992167 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:51.992235 31974 tablet_bootstrap.cc:492] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:51.992400 31974 ts_tablet_manager.cc:1367] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.059s user 0.023s sys 0.000s
I20240502 01:58:51.992810 31974 raft_consensus.cc:348] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.992918 31974 raft_consensus.cc:729] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:51.993053 31974 consensus_queue.cc:260] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:51.993795 31974 ts_tablet_manager.cc:1398] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:58:51.994249 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-dc99f8bfb06c4bcda6111b218125826c received from {username='slave'} at 127.19.43.198:56195
I20240502 01:58:51.994330 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-dc99f8bfb06c4bcda6111b218125826c on tablet dc99f8bfb06c4bcda6111b218125826c with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:52.008383 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 02e988cd07184e97921e7376da20f251 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.008639 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e988cd07184e97921e7376da20f251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.009109 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef794b6e73b4cc9bde557660033114d (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.009109 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9927f6676840dea245baa652c58b7d (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.009335 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef794b6e73b4cc9bde557660033114d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.009344 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9927f6676840dea245baa652c58b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.011304 31953 tablet_bootstrap.cc:492] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:52.012076 31953 tablet_bootstrap.cc:654] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.012895 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef794b6e73b4cc9bde557660033114d (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.013109 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef794b6e73b4cc9bde557660033114d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.014411 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 02e988cd07184e97921e7376da20f251 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.014636 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e988cd07184e97921e7376da20f251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.016732 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9927f6676840dea245baa652c58b7d (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.016945 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9927f6676840dea245baa652c58b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.017047 31997 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:52.018106 31997 tablet_bootstrap.cc:654] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.018901 31724 consensus_queue.cc:1035] T dd9b09843c784939bebe5695799e7916 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.020381 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9927f6676840dea245baa652c58b7d (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.020663 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9927f6676840dea245baa652c58b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.021232 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef794b6e73b4cc9bde557660033114d (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.021445 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef794b6e73b4cc9bde557660033114d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.022497 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 02e988cd07184e97921e7376da20f251 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.022729 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e988cd07184e97921e7376da20f251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.023524 31952 tablet_bootstrap.cc:492] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:52.024410 31952 tablet_bootstrap.cc:654] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.025067 31995 tablet_bootstrap.cc:492] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:52.025851 32002 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:52.026127 31995 tablet_bootstrap.cc:654] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.026818 32002 tablet_bootstrap.cc:654] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.027515 31997 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:52.027607 31997 ts_tablet_manager.cc:1367] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240502 01:58:52.028033 31997 raft_consensus.cc:348] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.028146 31997 raft_consensus.cc:374] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.028185 31997 raft_consensus.cc:729] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.028267 31995 tablet_bootstrap.cc:492] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:52.028312 31997 consensus_queue.cc:260] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.028384 31995 ts_tablet_manager.cc:1367] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:52.028597 31997 ts_tablet_manager.cc:1398] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:52.028889 31997 tablet_bootstrap.cc:492] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:52.029640 31997 tablet_bootstrap.cc:654] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.030997 31997 tablet_bootstrap.cc:492] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:52.031078 31997 ts_tablet_manager.cc:1367] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:52.031436 31997 raft_consensus.cc:348] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.031517 31997 raft_consensus.cc:374] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.031541 31997 raft_consensus.cc:729] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.031543 31995 raft_consensus.cc:348] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.031642 31995 raft_consensus.cc:374] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.031693 31995 raft_consensus.cc:729] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.031816 31997 consensus_queue.cc:260] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.031883 31995 consensus_queue.cc:260] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.032277 31997 ts_tablet_manager.cc:1398] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.033516 31724 raft_consensus.cc:482] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.033604 31724 raft_consensus.cc:504] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.033943 31724 leader_election.cc:288] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.034075 31952 tablet_bootstrap.cc:492] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:52.034106 31995 ts_tablet_manager.cc:1398] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:52.034170 31952 ts_tablet_manager.cc:1367] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:58:52.034587 31952 raft_consensus.cc:348] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.034739 31952 raft_consensus.cc:374] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.034790 31952 raft_consensus.cc:729] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.034907 31952 consensus_queue.cc:260] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.035058 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc35b3c50ed94ccfa748943f2cfeacf8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:52.035161 29488 raft_consensus.cc:2454] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:52.035202 31952 ts_tablet_manager.cc:1398] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.035380 31952 tablet_bootstrap.cc:492] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:52.035699 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc35b3c50ed94ccfa748943f2cfeacf8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:52.035818 26794 raft_consensus.cc:2454] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:52.035892 31995 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:52.036223 31952 tablet_bootstrap.cc:654] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.036492 29015 leader_election.cc:302] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:52.036881 31995 tablet_bootstrap.cc:654] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.037358 31724 raft_consensus.cc:2781] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.037412 31724 raft_consensus.cc:482] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.037443 31724 raft_consensus.cc:3037] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.039525 31953 tablet_bootstrap.cc:492] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:52.039605 31953 ts_tablet_manager.cc:1367] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20240502 01:58:52.040102 32002 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:52.040055 31953 raft_consensus.cc:348] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.040122 31952 tablet_bootstrap.cc:492] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:52.040148 31953 raft_consensus.cc:374] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.040176 31953 raft_consensus.cc:729] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.040182 32002 ts_tablet_manager.cc:1367] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:52.040205 31952 ts_tablet_manager.cc:1367] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:52.040289 31953 consensus_queue.cc:260] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.040551 31953 ts_tablet_manager.cc:1398] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.040608 31952 raft_consensus.cc:348] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.040691 31952 raft_consensus.cc:374] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.040660 32002 raft_consensus.cc:348] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.040721 31952 raft_consensus.cc:729] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.040732 31953 tablet_bootstrap.cc:492] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:52.040755 32002 raft_consensus.cc:374] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.040786 32002 raft_consensus.cc:729] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.040885 31952 consensus_queue.cc:260] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.040901 32002 consensus_queue.cc:260] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.041559 31953 tablet_bootstrap.cc:654] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.041565 32002 ts_tablet_manager.cc:1398] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.042750 31952 ts_tablet_manager.cc:1398] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:52.044236 31995 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:52.044374 31995 ts_tablet_manager.cc:1367] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:58:52.044951 31995 raft_consensus.cc:348] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.045001 31724 raft_consensus.cc:504] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.045059 31995 raft_consensus.cc:374] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.045092 31995 raft_consensus.cc:729] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.045285 31995 consensus_queue.cc:260] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.045320 31724 leader_election.cc:288] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.045670 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc35b3c50ed94ccfa748943f2cfeacf8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:52.045771 26794 raft_consensus.cc:3037] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.045912 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc35b3c50ed94ccfa748943f2cfeacf8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:52.045998 29488 raft_consensus.cc:3037] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.046370 31995 ts_tablet_manager.cc:1398] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:52.046682 26794 raft_consensus.cc:2454] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:52.046779 29488 raft_consensus.cc:2454] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:52.046972 29015 leader_election.cc:302] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:52.048869 31724 raft_consensus.cc:2781] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.049031 31724 raft_consensus.cc:686] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:52.049171 31724 consensus_queue.cc:237] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.051369 26510 catalog_manager.cc:5484] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.054273 31953 tablet_bootstrap.cc:492] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:52.054363 31953 ts_tablet_manager.cc:1367] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20240502 01:58:52.054703 31953 raft_consensus.cc:348] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.054785 31953 raft_consensus.cc:374] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.054963 31953 raft_consensus.cc:729] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.055075 31953 consensus_queue.cc:260] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.055361 31953 ts_tablet_manager.cc:1398] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:52.061554 30910 raft_consensus.cc:482] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.061642 30910 raft_consensus.cc:504] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.061975 30910 leader_election.cc:288] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:52.062546 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef794b6e73b4cc9bde557660033114d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:52.062546 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef794b6e73b4cc9bde557660033114d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:52.062659 29488 raft_consensus.cc:2454] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:52.062659 29213 raft_consensus.cc:2454] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:52.062960 31985 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.063074 26729 leader_election.cc:302] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:52.063344 30910 raft_consensus.cc:2781] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.063402 30910 raft_consensus.cc:482] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.063447 30910 raft_consensus.cc:3037] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.063553 31985 tablet_bootstrap.cc:492] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:52.063861 31985 ts_tablet_manager.cc:1367] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.121s user 0.021s sys 0.005s
I20240502 01:58:52.064337 30910 raft_consensus.cc:504] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.064312 31985 raft_consensus.cc:348] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:52.064416 31985 raft_consensus.cc:729] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.064556 31985 consensus_queue.cc:260] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:52.064630 30910 leader_election.cc:288] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:52.064886 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef794b6e73b4cc9bde557660033114d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:52.064915 31985 ts_tablet_manager.cc:1398] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.064980 29488 raft_consensus.cc:3037] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.065325 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-50bfc86904bc4347bd6beb7f16eb638d received from {username='slave'} at 127.19.43.199:38479
I20240502 01:58:52.065395 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-50bfc86904bc4347bd6beb7f16eb638d on tablet 50bfc86904bc4347bd6beb7f16eb638d with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:52.065721 29488 raft_consensus.cc:2454] T 9ef794b6e73b4cc9bde557660033114d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:52.065986 26730 leader_election.cc:302] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:52.066159 30910 raft_consensus.cc:2781] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.066300 30910 raft_consensus.cc:686] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:52.066406 30910 consensus_queue.cc:237] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.067132 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef794b6e73b4cc9bde557660033114d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:52.067250 29213 raft_consensus.cc:3037] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.068073 29213 raft_consensus.cc:2454] T 9ef794b6e73b4cc9bde557660033114d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:52.067998 26510 catalog_manager.cc:5484] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.069118 32009 raft_consensus.cc:482] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.069211 32009 raft_consensus.cc:504] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.069531 32009 leader_election.cc:288] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:52.069823 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e988cd07184e97921e7376da20f251" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:52.069923 29213 raft_consensus.cc:2454] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:52.069962 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e988cd07184e97921e7376da20f251" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:52.070118 29350 raft_consensus.cc:2454] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:52.070240 26861 leader_election.cc:302] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:52.070410 32009 raft_consensus.cc:2781] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.070458 32009 raft_consensus.cc:482] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.070498 32009 raft_consensus.cc:3037] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.071471 32009 raft_consensus.cc:504] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.071774 32009 leader_election.cc:288] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:52.071959 31895 raft_consensus.cc:482] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.072033 31895 raft_consensus.cc:504] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.072405 31895 leader_election.cc:288] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.072628 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9927f6676840dea245baa652c58b7d" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:52.072723 29488 raft_consensus.cc:2454] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:52.073272 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e988cd07184e97921e7376da20f251" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:52.073350 29286 leader_election.cc:302] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:52.073360 29350 raft_consensus.cc:3037] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.073599 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e988cd07184e97921e7376da20f251" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:52.073693 29213 raft_consensus.cc:3037] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.074123 31895 raft_consensus.cc:2781] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.074172 31895 raft_consensus.cc:482] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.074205 31895 raft_consensus.cc:3037] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.074402 29350 raft_consensus.cc:2454] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:52.075124 31895 raft_consensus.cc:504] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.075465 31895 leader_election.cc:288] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.075716 26859 leader_election.cc:302] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:52.075927 32009 raft_consensus.cc:2781] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.076017 32009 raft_consensus.cc:686] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:52.076128 32009 consensus_queue.cc:237] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.076164 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9927f6676840dea245baa652c58b7d" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:52.076242 29488 raft_consensus.cc:3037] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.076525 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9927f6676840dea245baa652c58b7d" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:52.076604 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9927f6676840dea245baa652c58b7d" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:52.076627 26794 raft_consensus.cc:2454] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:52.077039 29488 raft_consensus.cc:2454] T 1a9927f6676840dea245baa652c58b7d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:52.077306 29286 leader_election.cc:302] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters: b290c1bf893444078ed721f25995d124
I20240502 01:58:52.078055 29213 raft_consensus.cc:2454] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:52.078935 26509 catalog_manager.cc:5484] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:52.079245 31895 raft_consensus.cc:2781] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.079319 31895 raft_consensus.cc:686] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:52.079418 31895 consensus_queue.cc:237] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.081008 26510 catalog_manager.cc:5484] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.133225 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:52.134532 29195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:52.139987 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:52.140126 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:52.143162 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:52.143872 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:52.143982 29330 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:52.181267 29058 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:52.183732 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:52.189801 29488 raft_consensus.cc:1261] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.189806 26794 raft_consensus.cc:1261] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.190166 31991 consensus_queue.cc:1035] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.190328 31724 consensus_queue.cc:1035] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.192664 26513 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]). Expected version 1 got 0
I20240502 01:58:52.193015 26511 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]). Expected version 1 got 0
I20240502 01:58:52.195639 32072 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:52.195712 26509 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]). Expected version 1 got 0
I20240502 01:58:52.195946 32072 tablet.cc:1620] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:52.196167 26511 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7]). Expected version 1 got 0
I20240502 01:58:52.197949 32073 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:52.198277 32073 tablet.cc:1620] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:52.198684 32074 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:52.199036 32074 tablet.cc:1620] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:52.200610 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7] alter complete (version 1)
I20240502 01:58:52.261101 29079 consensus_queue.cc:237] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.262377 27063 raft_consensus.cc:1261] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.262562 29350 raft_consensus.cc:1261] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.262740 31724 consensus_queue.cc:1035] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:58:52.262933 29015 consensus_peers.cc:485] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6acba26a44294eb284d03176631ae39b. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.262952 32010 consensus_queue.cc:1035] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.264674 31724 raft_consensus.cc:2932] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.265062 29350 raft_consensus.cc:2932] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.267112 27063 raft_consensus.cc:2932] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.267252 26513 catalog_manager.cc:5484] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.286680 26661 consensus_queue.cc:237] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.288319 26927 raft_consensus.cc:1261] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.288393 27063 raft_consensus.cc:1261] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.288803 31069 consensus_queue.cc:1035] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.289155 31765 consensus_queue.cc:1035] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.290896 31765 raft_consensus.cc:2932] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.291276 26927 raft_consensus.cc:2932] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
W20240502 01:58:52.293308 26592 consensus_peers.cc:485] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99da1c4053674589b87a7898ed9d57de. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.293352 27063 raft_consensus.cc:2932] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.294171 26513 catalog_manager.cc:5484] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.303323 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:52.308115 32072 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:52.308442 32072 tablet.cc:1620] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:52.309063 32073 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:52.309319 32073 tablet.cc:1620] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:52.309584 32074 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:52.309872 32074 tablet.cc:1620] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:52.310407 26661 raft_consensus.cc:1203] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:52.315120 29213 raft_consensus.cc:1203] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:52.316496 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=2604e2bc1a194535a2d143e316aa70c7] alter complete (version 2)
I20240502 01:58:52.323673 29488 consensus_queue.cc:237] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.325178 29350 raft_consensus.cc:1261] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.325208 29213 raft_consensus.cc:1261] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.325588 32172 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.325840 31810 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.327829 31810 raft_consensus.cc:2932] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:52.329841 29213 raft_consensus.cc:2932] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:52.330122 29350 raft_consensus.cc:2932] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:52.330597 26512 catalog_manager.cc:5484] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:58:52.331674 29422 consensus_peers.cc:485] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee632f93fe234584814e6aa7b6a63bc0. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.348693 27063 consensus_queue.cc:237] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.350131 26794 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:52.350152 26995 consensus_peers.cc:485] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1950ace91824415b9bf3d1657bad72f. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.350368 26927 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.350530 32192 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.350715 31361 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.352324 31361 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.352715 26927 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.353883 26794 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.355683 26509 catalog_manager.cc:5484] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.373103 29079 consensus_queue.cc:237] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:52.374583 29213 raft_consensus.cc:1261] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.374673 26794 raft_consensus.cc:1261] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:58:52.374734 29016 consensus_peers.cc:485] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e66776cedd8407dad9c187b0b04f073. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.375015 31991 consensus_queue.cc:1035] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.375291 31724 consensus_queue.cc:1035] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.377177 31724 raft_consensus.cc:2932] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:52.377655 29213 raft_consensus.cc:2932] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:52.379797 26794 raft_consensus.cc:2932] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:52.379931 26512 catalog_manager.cc:5484] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:52.398988 29350 raft_consensus.cc:1203] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:52.399472 29079 consensus_queue.cc:237] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:52.400820 27063 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:52.400913 29488 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:58:52.401202 29013 consensus_peers.cc:485] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8afd4c5362464c4cba43d5ef28b50c29. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.401305 31991 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.401474 32213 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:52.403585 32213 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:52.403980 27063 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:52.405531 29488 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:52.407335 26511 catalog_manager.cc:5484] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:52.420815 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:52.425848 27063 consensus_queue.cc:237] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.426460 32072 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:52.427053 32074 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:52.427549 26794 raft_consensus.cc:1261] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:52.427773 26995 consensus_peers.cc:485] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e24e0bd4a1df4fed876e1c4c50af7d70. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.427842 26661 raft_consensus.cc:1261] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.428308 31362 consensus_queue.cc:1035] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.428496 32192 consensus_queue.cc:1035] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.431815 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=2604e2bc1a194535a2d143e316aa70c7] alter complete (version 3)
I20240502 01:58:52.432159 31361 raft_consensus.cc:2932] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.432540 26661 raft_consensus.cc:2932] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.435235 26509 catalog_manager.cc:5484] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.435842 32073 tablet.cc:1722] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:52.436134 26794 raft_consensus.cc:2932] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.453025 26794 consensus_queue.cc:237] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
W20240502 01:58:52.454584 26729 consensus_peers.cc:485] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e129aa347444eafab51845d9bf2ae32. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.454720 26927 raft_consensus.cc:1261] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.454800 26661 raft_consensus.cc:1261] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.455077 32024 consensus_queue.cc:1035] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.455248 32268 consensus_queue.cc:1035] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.456888 32024 raft_consensus.cc:2932] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:52.457270 26927 raft_consensus.cc:2932] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:52.458961 26661 raft_consensus.cc:2932] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:52.459488 26510 catalog_manager.cc:5484] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:52.475071 29489 raft_consensus.cc:1203] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:52.476106 32009 consensus_queue.cc:1035] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.479929 26927 consensus_queue.cc:237] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.481325 27064 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:52.481392 26859 consensus_peers.cc:485] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5515469fdf494900ae20f20fdfca587c. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:52.481514 26661 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:52.481779 32009 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.481848 32289 consensus_queue.cc:1035] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.481928 31715 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:52.484117 32009 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.484464 26661 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.485798 27064 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.486583 26510 catalog_manager.cc:5484] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:52.508422 30910 consensus_queue.cc:1035] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.511266 30910 consensus_queue.cc:1035] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.539508 26515 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:52.539630 26515 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:52.541642 26515 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet fc35b3c50ed94ccfa748943f2cfeacf8
I20240502 01:58:52.542299 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet fc35b3c50ed94ccfa748943f2cfeacf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:52 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:52.542544 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet fc35b3c50ed94ccfa748943f2cfeacf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:52 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:52.542915 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet fc35b3c50ed94ccfa748943f2cfeacf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:52 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:52.543733 32338 tablet_replica.cc:330] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:52.543969 32339 tablet_replica.cc:330] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:52.544171 32340 tablet_replica.cc:330] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:52.544356 32340 raft_consensus.cc:2229] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:52.544546 32339 raft_consensus.cc:2229] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:52.544682 32339 raft_consensus.cc:2258] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:52.544898 32338 raft_consensus.cc:2229] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:52.545171 32340 raft_consensus.cc:2258] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:52.545346 32338 raft_consensus.cc:2258] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:52.548058 32338 ts_tablet_manager.cc:1875] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:52.548213 32339 ts_tablet_manager.cc:1875] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:52.548629 32340 ts_tablet_manager.cc:1875] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:52.552904 32340 ts_tablet_manager.cc:1888] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:52.552969 32340 log.cc:1196] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/fc35b3c50ed94ccfa748943f2cfeacf8
I20240502 01:58:52.553268 32340 ts_tablet_manager.cc:1909] T fc35b3c50ed94ccfa748943f2cfeacf8 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:52.553862 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table_new_name [id=2604e2bc1a194535a2d143e316aa70c7]) successfully deleted
I20240502 01:58:52.555249 32338 ts_tablet_manager.cc:1888] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:52.555378 32338 log.cc:1196] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/fc35b3c50ed94ccfa748943f2cfeacf8
I20240502 01:58:52.556715 32339 ts_tablet_manager.cc:1888] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:52.556838 32339 log.cc:1196] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/fc35b3c50ed94ccfa748943f2cfeacf8
I20240502 01:58:52.557556 32339 ts_tablet_manager.cc:1909] T fc35b3c50ed94ccfa748943f2cfeacf8 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:52.558365 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table_new_name [id=2604e2bc1a194535a2d143e316aa70c7]) successfully deleted
I20240502 01:58:52.558827 32338 ts_tablet_manager.cc:1909] T fc35b3c50ed94ccfa748943f2cfeacf8 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:52.559516 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet fc35b3c50ed94ccfa748943f2cfeacf8 (table rebalancer_dynamic_table_new_name [id=2604e2bc1a194535a2d143e316aa70c7]) successfully deleted
I20240502 01:58:52.569581 29330 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:52.577337 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:52.579422 29195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:52.588491 29058 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:52.590303 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:52.593516 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:52.609792 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:52.610092 26641 tablet_service.cc:1467] Processing CreateTablet for tablet b01685bedb5a41bab08b74899e77197c (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.610126 29058 tablet_service.cc:1467] Processing CreateTablet for tablet b01685bedb5a41bab08b74899e77197c (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.610359 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01685bedb5a41bab08b74899e77197c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.610360 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01685bedb5a41bab08b74899e77197c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.612872 29454 tablet_service.cc:1467] Processing CreateTablet for tablet b01685bedb5a41bab08b74899e77197c (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.613093 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01685bedb5a41bab08b74899e77197c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.613313 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 68f002fcea6f4378954808dbfb5f10e6 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.613363 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac59b967e484772b3366b919757ced7 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.613538 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f002fcea6f4378954808dbfb5f10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.613698 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac59b967e484772b3366b919757ced7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.615769 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 68f002fcea6f4378954808dbfb5f10e6 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.616020 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f002fcea6f4378954808dbfb5f10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.617365 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac59b967e484772b3366b919757ced7 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.617667 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac59b967e484772b3366b919757ced7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.618412 32357 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:52.618631 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac59b967e484772b3366b919757ced7 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.618896 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac59b967e484772b3366b919757ced7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.619611 32357 tablet_bootstrap.cc:654] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.620661 32357 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:52.620747 32357 ts_tablet_manager.cc:1367] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:52.621192 32357 raft_consensus.cc:348] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.621289 32357 raft_consensus.cc:374] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.621315 32357 raft_consensus.cc:729] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.618628 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 68f002fcea6f4378954808dbfb5f10e6 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=66b1eea6466f43a087b8d0a822bf5f3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:52.621657 32357 consensus_queue.cc:260] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.621861 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f002fcea6f4378954808dbfb5f10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.622154 32357 ts_tablet_manager.cc:1398] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:52.625705 32354 tablet_bootstrap.cc:492] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:52.628556 26794 raft_consensus.cc:3037] T 1a9927f6676840dea245baa652c58b7d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.628832 32359 tablet_bootstrap.cc:492] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:52.628937 32355 tablet_bootstrap.cc:492] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:52.629827 32359 tablet_bootstrap.cc:654] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.629863 32355 tablet_bootstrap.cc:654] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.629786 31620 consensus_queue.cc:1035] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.630178 32353 tablet_bootstrap.cc:492] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:52.630427 32356 tablet_bootstrap.cc:492] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:52.631317 32359 tablet_bootstrap.cc:492] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:52.631332 32356 tablet_bootstrap.cc:654] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.631469 32359 ts_tablet_manager.cc:1367] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:52.632179 32359 raft_consensus.cc:348] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.633270 32359 raft_consensus.cc:374] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.633358 32359 raft_consensus.cc:729] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.633564 32359 consensus_queue.cc:260] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.635882 32359 ts_tablet_manager.cc:1398] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:52.636128 32359 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:52.637876 32359 tablet_bootstrap.cc:654] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.639313 32359 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:52.639564 32359 ts_tablet_manager.cc:1367] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:52.640149 32359 raft_consensus.cc:348] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.641049 31620 consensus_queue.cc:1035] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:52.640293 32359 raft_consensus.cc:374] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.642473 32359 raft_consensus.cc:729] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.643251 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:52.642679 32359 consensus_queue.cc:260] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.643848 32359 ts_tablet_manager.cc:1398] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:52.652837 29195 tablet_service.cc:1467] Processing CreateTablet for tablet fe68fe7c69554cbeb9984a7be858bb26 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]), partition=
I20240502 01:58:52.653048 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe68fe7c69554cbeb9984a7be858bb26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.653034 26774 tablet_service.cc:1467] Processing CreateTablet for tablet fe68fe7c69554cbeb9984a7be858bb26 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]), partition=
I20240502 01:58:52.653205 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe68fe7c69554cbeb9984a7be858bb26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.654996 32357 tablet_bootstrap.cc:492] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:52.655778 26909 tablet_service.cc:1467] Processing CreateTablet for tablet fe68fe7c69554cbeb9984a7be858bb26 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]), partition=
I20240502 01:58:52.655814 32357 tablet_bootstrap.cc:654] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.656003 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe68fe7c69554cbeb9984a7be858bb26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.656751 32357 tablet_bootstrap.cc:492] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:52.656831 32357 ts_tablet_manager.cc:1367] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:52.657217 32357 raft_consensus.cc:348] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.657302 32357 raft_consensus.cc:374] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.657328 32357 raft_consensus.cc:729] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.657433 32357 consensus_queue.cc:260] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.657660 32357 ts_tablet_manager.cc:1398] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:52.658617 32354 tablet_bootstrap.cc:654] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.659564 32353 tablet_bootstrap.cc:654] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.659852 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:52.660439 32356 tablet_bootstrap.cc:492] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:52.660526 32356 ts_tablet_manager.cc:1367] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240502 01:58:52.660950 32356 raft_consensus.cc:348] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.661051 32356 raft_consensus.cc:374] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.661075 32356 raft_consensus.cc:729] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.661360 32356 consensus_queue.cc:260] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.661849 32353 tablet_bootstrap.cc:492] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:52.661967 32353 ts_tablet_manager.cc:1367] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 01:58:52.662406 32353 raft_consensus.cc:348] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.663879 32354 tablet_bootstrap.cc:492] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:52.664007 32354 ts_tablet_manager.cc:1367] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20240502 01:58:52.664662 32354 raft_consensus.cc:348] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.664786 32354 raft_consensus.cc:374] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.664819 32354 raft_consensus.cc:729] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.665038 32354 consensus_queue.cc:260] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.666363 32354 ts_tablet_manager.cc:1398] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:52.666568 32354 tablet_bootstrap.cc:492] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:52.668828 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:52.670293 32368 tablet_bootstrap.cc:492] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:52.671175 32368 tablet_bootstrap.cc:654] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.672245 32368 tablet_bootstrap.cc:492] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:52.672387 32368 ts_tablet_manager.cc:1367] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:52.672915 32368 raft_consensus.cc:348] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.673004 32368 raft_consensus.cc:374] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.673032 32368 raft_consensus.cc:729] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.673167 32368 consensus_queue.cc:260] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.673437 32368 ts_tablet_manager.cc:1398] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.678910 32356 ts_tablet_manager.cc:1398] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.018s user 0.001s sys 0.000s
I20240502 01:58:52.679008 32353 raft_consensus.cc:374] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.679076 32353 raft_consensus.cc:729] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.679240 32353 consensus_queue.cc:260] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.679512 32356 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:52.681910 32354 tablet_bootstrap.cc:654] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.682803 32353 ts_tablet_manager.cc:1398] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.021s user 0.001s sys 0.000s
I20240502 01:58:52.683524 32356 tablet_bootstrap.cc:654] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.683928 32354 tablet_bootstrap.cc:492] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:52.684036 32354 ts_tablet_manager.cc:1367] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 01:58:52.684729 32354 raft_consensus.cc:348] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.684964 32354 raft_consensus.cc:374] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.685016 32354 raft_consensus.cc:729] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.685019 32355 tablet_bootstrap.cc:492] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:52.685114 32355 ts_tablet_manager.cc:1367] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.000s
I20240502 01:58:52.685137 32354 consensus_queue.cc:260] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.685415 32354 ts_tablet_manager.cc:1398] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.685572 32355 raft_consensus.cc:348] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.687515 32355 raft_consensus.cc:374] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.687606 32355 raft_consensus.cc:729] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.687767 32355 consensus_queue.cc:260] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.688191 32355 ts_tablet_manager.cc:1398] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:52.692838 32356 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:52.692917 32356 ts_tablet_manager.cc:1367] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 01:58:52.693259 32356 raft_consensus.cc:348] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.693356 32356 raft_consensus.cc:374] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.693387 32356 raft_consensus.cc:729] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.693540 32356 consensus_queue.cc:260] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.693807 32356 ts_tablet_manager.cc:1398] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.693992 32356 tablet_bootstrap.cc:492] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:52.695572 32356 tablet_bootstrap.cc:654] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:52.696696 32356 tablet_bootstrap.cc:492] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:52.696775 32356 ts_tablet_manager.cc:1367] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:52.697144 32356 raft_consensus.cc:348] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.697227 32356 raft_consensus.cc:374] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:52.697259 32356 raft_consensus.cc:729] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.697356 32356 consensus_queue.cc:260] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.697568 32356 ts_tablet_manager.cc:1398] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:52.730908 32394 ts_tablet_manager.cc:902] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:52.731415 32394 tablet_copy_client.cc:308] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:52.731791 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet e1950ace91824415b9bf3d1657bad72f from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:55881)
I20240502 01:58:52.731886 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet e1950ace91824415b9bf3d1657bad72f from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:55881: session id = f3d18a019013487ab801599cfb80e4b4-e1950ace91824415b9bf3d1657bad72f
I20240502 01:58:52.732899 27083 tablet_copy_source_session.cc:205] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.733701 32394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1950ace91824415b9bf3d1657bad72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.735838 32394 tablet_copy_client.cc:783] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.736038 32394 tablet_copy_client.cc:647] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.737146 32394 tablet_copy_client.cc:523] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.738286 32394 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:52.746933 32427 ts_tablet_manager.cc:902] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:52.747503 32427 tablet_copy_client.cc:308] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:58:52.747968 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet 99da1c4053674589b87a7898ed9d57de from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:39103)
I20240502 01:58:52.748067 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet 99da1c4053674589b87a7898ed9d57de from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:39103: session id = f3d18a019013487ab801599cfb80e4b4-99da1c4053674589b87a7898ed9d57de
I20240502 01:58:52.749054 26682 tablet_copy_source_session.cc:205] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.749866 32427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99da1c4053674589b87a7898ed9d57de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.752234 32427 tablet_copy_client.cc:783] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.752416 32427 tablet_copy_client.cc:647] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.755447 32427 tablet_copy_client.cc:523] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.756502 32427 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:52.762810 32394 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.763054 32394 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:52.763200 32394 ts_tablet_manager.cc:1367] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.000s
I20240502 01:58:52.763549 32394 raft_consensus.cc:348] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.763648 32394 raft_consensus.cc:729] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.763792 32394 consensus_queue.cc:260] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.764159 32394 ts_tablet_manager.cc:1398] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:58:52.764524 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-e1950ace91824415b9bf3d1657bad72f received from {username='slave'} at 127.19.43.197:55881
I20240502 01:58:52.764591 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-e1950ace91824415b9bf3d1657bad72f on tablet e1950ace91824415b9bf3d1657bad72f with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:52.768775 32213 raft_consensus.cc:482] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.768855 32213 raft_consensus.cc:504] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.769158 32213 leader_election.cc:288] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:52.769744 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01685bedb5a41bab08b74899e77197c" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:52.769881 26661 raft_consensus.cc:2454] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:52.770202 29015 leader_election.cc:302] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:52.770373 32213 raft_consensus.cc:2781] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.770442 32213 raft_consensus.cc:482] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.770484 32213 raft_consensus.cc:3037] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.770912 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01685bedb5a41bab08b74899e77197c" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:52.771046 29489 raft_consensus.cc:2454] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:52.771379 32213 raft_consensus.cc:504] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.771713 32213 leader_election.cc:288] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:52.772102 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01685bedb5a41bab08b74899e77197c" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:52.772151 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01685bedb5a41bab08b74899e77197c" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:52.772209 26661 raft_consensus.cc:3037] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.772255 29489 raft_consensus.cc:3037] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.773102 26661 raft_consensus.cc:2454] T b01685bedb5a41bab08b74899e77197c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:52.773104 29489 raft_consensus.cc:2454] T b01685bedb5a41bab08b74899e77197c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:52.773380 29015 leader_election.cc:302] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:52.773916 32213 raft_consensus.cc:2781] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.774003 32213 raft_consensus.cc:686] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:52.774139 32213 consensus_queue.cc:237] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:52.775969 26509 catalog_manager.cc:5484] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:52.784407 32427 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.784627 32427 tablet_bootstrap.cc:492] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:52.784773 32427 ts_tablet_manager.cc:1367] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.000s
I20240502 01:58:52.785096 32427 raft_consensus.cc:348] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.785200 32427 raft_consensus.cc:729] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.785323 32427 consensus_queue.cc:260] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:52.785578 32427 ts_tablet_manager.cc:1398] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:52.786087 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-99da1c4053674589b87a7898ed9d57de received from {username='slave'} at 127.19.43.197:39103
I20240502 01:58:52.786170 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-99da1c4053674589b87a7898ed9d57de on tablet 99da1c4053674589b87a7898ed9d57de with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:52.811024 32434 ts_tablet_manager.cc:902] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:52.811805 32434 tablet_copy_client.cc:308] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:58:52.811945 31391 raft_consensus.cc:482] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.812047 31391 raft_consensus.cc:504] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.812450 31391 leader_election.cc:288] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:52.812727 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet 8e66776cedd8407dad9c187b0b04f073 from peer 68e5b1eeb04448fea818a14991eab87c ({username='slave'} at 127.19.43.196:54885)
I20240502 01:58:52.812845 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet 8e66776cedd8407dad9c187b0b04f073 from peer 68e5b1eeb04448fea818a14991eab87c at {username='slave'} at 127.19.43.196:54885: session id = 68e5b1eeb04448fea818a14991eab87c-8e66776cedd8407dad9c187b0b04f073
I20240502 01:58:52.812898 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac59b967e484772b3366b919757ced7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:52.813014 29350 raft_consensus.cc:2454] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:52.813243 29147 leader_election.cc:302] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:52.813386 31391 raft_consensus.cc:2781] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.813429 31391 raft_consensus.cc:482] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.813452 31391 raft_consensus.cc:3037] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.814256 29102 tablet_copy_source_session.cc:205] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.814419 31391 raft_consensus.cc:504] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.814716 31391 leader_election.cc:288] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:58:52.814869 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac59b967e484772b3366b919757ced7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:52.814989 26661 raft_consensus.cc:2454] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:52.815829 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac59b967e484772b3366b919757ced7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:52.815934 26661 raft_consensus.cc:3037] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.816376 32434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e66776cedd8407dad9c187b0b04f073. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.816705 26661 raft_consensus.cc:2454] T 6ac59b967e484772b3366b919757ced7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:52.817088 29149 leader_election.cc:302] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:52.817293 31757 raft_consensus.cc:2781] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.817451 31757 raft_consensus.cc:686] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:52.817582 31757 consensus_queue.cc:237] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:52.818346 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac59b967e484772b3366b919757ced7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:52.818465 29349 raft_consensus.cc:3037] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.818768 32434 tablet_copy_client.cc:783] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.818907 32434 tablet_copy_client.cc:647] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.819376 29349 raft_consensus.cc:2454] T 6ac59b967e484772b3366b919757ced7 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:52.819377 26509 catalog_manager.cc:5484] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:52.820618 32434 tablet_copy_client.cc:523] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.821998 32434 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:52.831715 32434 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.831956 32434 tablet_bootstrap.cc:492] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap complete.
I20240502 01:58:52.832093 32434 ts_tablet_manager.cc:1367] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20240502 01:58:52.832515 32434 raft_consensus.cc:348] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:52.832623 32434 raft_consensus.cc:729] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.832758 32434 consensus_queue.cc:260] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:52.833101 32434 ts_tablet_manager.cc:1398] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:52.833673 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session 68e5b1eeb04448fea818a14991eab87c-8e66776cedd8407dad9c187b0b04f073 received from {username='slave'} at 127.19.43.196:54885
I20240502 01:58:52.833756 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session 68e5b1eeb04448fea818a14991eab87c-8e66776cedd8407dad9c187b0b04f073 on tablet 8e66776cedd8407dad9c187b0b04f073 with peer 68e5b1eeb04448fea818a14991eab87c
I20240502 01:58:52.835572 31391 raft_consensus.cc:482] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.835651 31391 raft_consensus.cc:504] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.836009 31391 leader_election.cc:288] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.836254 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f002fcea6f4378954808dbfb5f10e6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:52.836357 29349 raft_consensus.cc:2454] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:52.836628 32442 ts_tablet_manager.cc:902] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:52.836678 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f002fcea6f4378954808dbfb5f10e6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:52.836804 26794 raft_consensus.cc:2454] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:52.836869 29147 leader_election.cc:302] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:52.837023 31391 raft_consensus.cc:2781] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.837074 31391 raft_consensus.cc:482] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.837113 31391 raft_consensus.cc:3037] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.837417 32442 tablet_copy_client.cc:308] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:58:52.837908 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet 6acba26a44294eb284d03176631ae39b from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:41057)
I20240502 01:58:52.837996 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet 6acba26a44294eb284d03176631ae39b from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:41057: session id = b290c1bf893444078ed721f25995d124-6acba26a44294eb284d03176631ae39b
I20240502 01:58:52.837980 31391 raft_consensus.cc:504] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.838306 31391 leader_election.cc:288] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.838559 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f002fcea6f4378954808dbfb5f10e6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:52.838584 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68f002fcea6f4378954808dbfb5f10e6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:52.838656 26794 raft_consensus.cc:3037] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.838677 29349 raft_consensus.cc:3037] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.839007 29102 tablet_copy_source_session.cc:205] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.839494 26794 raft_consensus.cc:2454] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:52.839560 29349 raft_consensus.cc:2454] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:52.839823 29149 leader_election.cc:302] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:52.840020 31391 raft_consensus.cc:2781] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.840101 31391 raft_consensus.cc:686] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:52.840150 32442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acba26a44294eb284d03176631ae39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.840224 31391 consensus_queue.cc:237] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.842072 26509 catalog_manager.cc:5484] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.842525 32442 tablet_copy_client.cc:783] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.842703 32442 tablet_copy_client.cc:647] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.844375 32442 tablet_copy_client.cc:523] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.845480 32442 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:52.849086 32442 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.849310 32442 tablet_bootstrap.cc:492] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:58:52.849444 32442 ts_tablet_manager.cc:1367] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240502 01:58:52.849822 32442 raft_consensus.cc:348] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.849905 32442 raft_consensus.cc:729] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.850011 32442 consensus_queue.cc:260] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.850289 32442 ts_tablet_manager.cc:1398] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 01:58:52.850744 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session b290c1bf893444078ed721f25995d124-6acba26a44294eb284d03176631ae39b received from {username='slave'} at 127.19.43.194:41057
I20240502 01:58:52.850814 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session b290c1bf893444078ed721f25995d124-6acba26a44294eb284d03176631ae39b on tablet 6acba26a44294eb284d03176631ae39b with peer b290c1bf893444078ed721f25995d124
I20240502 01:58:52.874711 31757 raft_consensus.cc:482] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:52.874786 31757 raft_consensus.cc:504] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.875120 31757 leader_election.cc:288] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.875432 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe68fe7c69554cbeb9984a7be858bb26" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:52.875545 26794 raft_consensus.cc:2454] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:52.875794 29149 leader_election.cc:302] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:52.875944 31757 raft_consensus.cc:2781] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:52.875993 31757 raft_consensus.cc:482] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:52.876029 31757 raft_consensus.cc:3037] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.876731 31757 raft_consensus.cc:504] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.877040 31757 leader_election.cc:288] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.877447 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe68fe7c69554cbeb9984a7be858bb26" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:52.877561 26794 raft_consensus.cc:3037] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:52.878448 26794 raft_consensus.cc:2454] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:52.878736 29149 leader_election.cc:302] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:52.878751 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe68fe7c69554cbeb9984a7be858bb26" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:52.878814 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe68fe7c69554cbeb9984a7be858bb26" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:52.878865 26927 raft_consensus.cc:2454] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:52.878907 31757 raft_consensus.cc:2781] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:52.878969 31757 raft_consensus.cc:686] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:52.879076 31757 consensus_queue.cc:237] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:52.880620 26512 catalog_manager.cc:5484] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:52.890540 32462 ts_tablet_manager.cc:902] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:52.891656 32462 tablet_copy_client.cc:308] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:58:52.892931 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 2e129aa347444eafab51845d9bf2ae32 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:58:52.893025 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 2e129aa347444eafab51845d9bf2ae32 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-2e129aa347444eafab51845d9bf2ae32
I20240502 01:58:52.893940 26815 tablet_copy_source_session.cc:205] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.896119 29330 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:52.896533 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:52.897384 32462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e129aa347444eafab51845d9bf2ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.900696 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:52.900987 32462 tablet_copy_client.cc:783] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.901115 32462 tablet_copy_client.cc:647] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.905797 29058 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:52.915992 32462 tablet_copy_client.cc:523] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.916678 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:52.917646 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:52.919620 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:52.918227 32462 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:52.930286 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:52.942528 32467 ts_tablet_manager.cc:902] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:52.943727 32467 tablet_copy_client.cc:308] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:58:52.944186 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet 8afd4c5362464c4cba43d5ef28b50c29 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:34855)
I20240502 01:58:52.944278 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet 8afd4c5362464c4cba43d5ef28b50c29 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:34855: session id = 468dd27456c64d67b22e8a58af93d93d-8afd4c5362464c4cba43d5ef28b50c29
I20240502 01:58:52.944950 29102 tablet_copy_source_session.cc:205] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.945726 32467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8afd4c5362464c4cba43d5ef28b50c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.947988 32467 tablet_copy_client.cc:783] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.948176 32467 tablet_copy_client.cc:647] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.951321 32467 tablet_copy_client.cc:523] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.952476 32467 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:52.954645 32427 ts_tablet_manager.cc:902] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:52.954943 32473 ts_tablet_manager.cc:902] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:52.955026 32427 tablet_copy_client.cc:308] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:52.955755 32473 tablet_copy_client.cc:308] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.200:36259
I20240502 01:58:52.956491 29512 tablet_copy_service.cc:140] P 7c693d0933c64d9babc07f404df45cd1: Received BeginTabletCopySession request for tablet ee632f93fe234584814e6aa7b6a63bc0 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:53915)
I20240502 01:58:52.956599 29512 tablet_copy_service.cc:161] P 7c693d0933c64d9babc07f404df45cd1: Beginning new tablet copy session on tablet ee632f93fe234584814e6aa7b6a63bc0 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:53915: session id = 842c1b011b5d4c77bfd9b9591111502e-ee632f93fe234584814e6aa7b6a63bc0
I20240502 01:58:52.957571 29512 tablet_copy_source_session.cc:205] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.959098 32473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee632f93fe234584814e6aa7b6a63bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.962764 32473 tablet_copy_client.cc:783] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.962973 32473 tablet_copy_client.cc:647] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.966917 32473 tablet_copy_client.cc:523] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.968360 32473 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:52.972653 32394 ts_tablet_manager.cc:902] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:52.973098 32394 tablet_copy_client.cc:308] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:52.973588 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet e24e0bd4a1df4fed876e1c4c50af7d70 from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:55881)
I20240502 01:58:52.973682 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet e24e0bd4a1df4fed876e1c4c50af7d70 from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:55881: session id = f3d18a019013487ab801599cfb80e4b4-e24e0bd4a1df4fed876e1c4c50af7d70
I20240502 01:58:52.974570 27083 tablet_copy_source_session.cc:205] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.974876 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 5515469fdf494900ae20f20fdfca587c from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:46065)
I20240502 01:58:52.974954 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 5515469fdf494900ae20f20fdfca587c from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:46065: session id = f3d18a019013487ab801599cfb80e4b4-5515469fdf494900ae20f20fdfca587c
I20240502 01:58:52.975767 26950 tablet_copy_source_session.cc:205] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:52.976857 32394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24e0bd4a1df4fed876e1c4c50af7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.978906 32394 tablet_copy_client.cc:783] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.979025 32394 tablet_copy_client.cc:647] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.979365 32427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5515469fdf494900ae20f20fdfca587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:52.981272 32427 tablet_copy_client.cc:783] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:52.981386 32427 tablet_copy_client.cc:647] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:52.981731 32462 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.981983 32462 tablet_bootstrap.cc:492] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:52.982161 32462 ts_tablet_manager.cc:1367] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.064s user 0.024s sys 0.000s
I20240502 01:58:52.983522 32462 raft_consensus.cc:348] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:52.983728 32462 raft_consensus.cc:729] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.983927 32462 consensus_queue.cc:260] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:52.984344 32473 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.984364 32462 ts_tablet_manager.cc:1398] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:52.984596 32473 tablet_bootstrap.cc:492] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:58:52.985025 32473 ts_tablet_manager.cc:1367] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20240502 01:58:52.985051 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-2e129aa347444eafab51845d9bf2ae32 received from {username='slave'} at 127.19.43.198:56195
I20240502 01:58:52.985112 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-2e129aa347444eafab51845d9bf2ae32 on tablet 2e129aa347444eafab51845d9bf2ae32 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:52.985579 32473 raft_consensus.cc:348] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.986081 32473 raft_consensus.cc:729] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.986505 32473 consensus_queue.cc:260] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:52.986550 32427 tablet_copy_client.cc:523] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.986961 32473 ts_tablet_manager.cc:1398] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:52.987653 32427 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:52.991626 29512 tablet_copy_service.cc:342] P 7c693d0933c64d9babc07f404df45cd1: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-ee632f93fe234584814e6aa7b6a63bc0 received from {username='slave'} at 127.19.43.195:53915
I20240502 01:58:52.991724 29512 tablet_copy_service.cc:433] P 7c693d0933c64d9babc07f404df45cd1: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-ee632f93fe234584814e6aa7b6a63bc0 on tablet ee632f93fe234584814e6aa7b6a63bc0 with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:58:52.992983 32467 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:52.993206 32467 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:52.993352 32467 ts_tablet_manager.cc:1367] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.000s
I20240502 01:58:52.993713 32467 raft_consensus.cc:348] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:52.993808 32467 raft_consensus.cc:729] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:52.993924 32467 consensus_queue.cc:260] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:52.994196 32467 ts_tablet_manager.cc:1398] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:52.994637 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-8afd4c5362464c4cba43d5ef28b50c29 received from {username='slave'} at 127.19.43.199:34855
I20240502 01:58:52.994707 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-8afd4c5362464c4cba43d5ef28b50c29 on tablet 8afd4c5362464c4cba43d5ef28b50c29 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:52.995749 32394 tablet_copy_client.cc:523] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:52.996822 32394 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:52.998879 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:53.002970 26927 raft_consensus.cc:3037] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.003829 26927 raft_consensus.cc:1261] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:53.004138 26794 raft_consensus.cc:1261] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:53.004196 31757 consensus_queue.cc:1035] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.004948 31757 consensus_queue.cc:1035] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.007267 26512 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]). Expected version 1 got 0
I20240502 01:58:53.008428 26509 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]). Expected version 1 got 0
I20240502 01:58:53.009060 32292 mvcc.cc:205] Tried to move back new op lower bound from 7023063584776826880 to 7023063584273928192. Current Snapshot: MvccSnapshot[applied={T|T < 7023063584776826880}]
I20240502 01:58:53.009755 32506 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:53.010110 32506 tablet.cc:1620] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:53.011171 26509 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]). Expected version 1 got 0
I20240502 01:58:53.018702 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44] alter complete (version 1)
I20240502 01:58:53.020021 32509 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:53.020328 32505 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:53.020390 32509 tablet.cc:1620] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:53.020668 32505 tablet.cc:1620] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:53.023469 26512 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44]). Expected version 1 got 0
I20240502 01:58:53.025467 32427 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:53.025760 32427 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:53.025939 32427 ts_tablet_manager.cc:1367] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.038s user 0.025s sys 0.000s
I20240502 01:58:53.026434 32427 raft_consensus.cc:348] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.026610 32427 raft_consensus.cc:729] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.026813 32427 consensus_queue.cc:260] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.027205 32427 ts_tablet_manager.cc:1398] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:53.027664 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-5515469fdf494900ae20f20fdfca587c received from {username='slave'} at 127.19.43.197:46065
I20240502 01:58:53.027748 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-5515469fdf494900ae20f20fdfca587c on tablet 5515469fdf494900ae20f20fdfca587c with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:53.036553 32394 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:53.036772 32394 tablet_bootstrap.cc:492] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:58:53.036931 32394 ts_tablet_manager.cc:1367] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.040s user 0.019s sys 0.000s
I20240502 01:58:53.037369 32394 raft_consensus.cc:348] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.037454 32394 raft_consensus.cc:729] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.037564 32394 consensus_queue.cc:260] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.037811 32394 ts_tablet_manager.cc:1398] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:53.038273 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-e24e0bd4a1df4fed876e1c4c50af7d70 received from {username='slave'} at 127.19.43.197:55881
I20240502 01:58:53.038347 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-e24e0bd4a1df4fed876e1c4c50af7d70 on tablet e24e0bd4a1df4fed876e1c4c50af7d70 with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:53.088894 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:53.091861 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:53.092527 29330 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:53.099151 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:53.100183 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:53.100658 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:53.103418 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:53.124722 29058 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:53.127848 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:53.135385 32505 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:53.135677 32505 tablet.cc:1620] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:53.136327 32509 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:53.136608 32509 tablet.cc:1620] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:53.139355 32506 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:53.139648 32506 tablet.cc:1620] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:53.142711 26514 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=34123f56c6cc4db1ba26e43450c24d44] alter complete (version 2)
I20240502 01:58:53.166395 29079 raft_consensus.cc:1203] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:53.189775 31991 consensus_queue.cc:1035] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.193217 31991 consensus_queue.cc:1035] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.202754 29079 raft_consensus.cc:1203] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:53.210007 27063 raft_consensus.cc:1203] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:53.247399 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:53.251492 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:53.252092 29330 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:53.255053 32505 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:53.256697 29058 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:53.259466 32509 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:53.264791 32506 tablet.cc:1722] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:53.264870 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:53.265074 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=34123f56c6cc4db1ba26e43450c24d44] alter complete (version 3)
I20240502 01:58:53.265319 29454 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 01:58:53.266180 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:53.270246 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:53.282784 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:53.302609 26794 raft_consensus.cc:1203] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:53.342778 27063 consensus_queue.cc:237] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.343902 26927 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:53.343958 29079 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:53.344262 32573 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:53.344394 32192 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:53.345897 32192 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:53.346477 29079 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:53.347491 26927 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:53.348166 29216 raft_consensus.cc:1203] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:53.348791 26515 catalog_manager.cc:5484] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:53.351883 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet e1950ace91824415b9bf3d1657bad72f with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:53.352190 32627 tablet_replica.cc:330] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:53.352329 32627 raft_consensus.cc:2229] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:53.352605 32627 raft_consensus.cc:2258] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:53.353204 32627 ts_tablet_manager.cc:1875] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:53.355368 32627 ts_tablet_manager.cc:1888] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:53.355446 32627 log.cc:1196] T e1950ace91824415b9bf3d1657bad72f P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/e1950ace91824415b9bf3d1657bad72f
I20240502 01:58:53.355965 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet e1950ace91824415b9bf3d1657bad72f (table rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]) successfully deleted
I20240502 01:58:53.370913 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:53.375582 26512 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:53.375689 26512 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:53.377264 29330 tablet_service.cc:1467] Processing CreateTablet for tablet aa8dc7e708a94f0b9947e1b420f1fa40 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.377563 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8dc7e708a94f0b9947e1b420f1fa40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.377617 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 59c5307ea58d40dc80079b45f17db1d2 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.377892 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c5307ea58d40dc80079b45f17db1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.377944 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 59c5307ea58d40dc80079b45f17db1d2 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.378255 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c5307ea58d40dc80079b45f17db1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.378225 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc3e48cb92b4788a88c3cbf715d1596 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.378412 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 59c5307ea58d40dc80079b45f17db1d2 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.378474 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc3e48cb92b4788a88c3cbf715d1596. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.378645 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c5307ea58d40dc80079b45f17db1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.380019 29059 tablet_service.cc:1467] Processing CreateTablet for tablet aa8dc7e708a94f0b9947e1b420f1fa40 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.380225 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8dc7e708a94f0b9947e1b420f1fa40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.380762 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc3e48cb92b4788a88c3cbf715d1596 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.381037 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc3e48cb92b4788a88c3cbf715d1596. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.382825 32643 tablet_bootstrap.cc:492] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:53.383297 32644 tablet_bootstrap.cc:492] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:53.384024 32643 tablet_bootstrap.cc:654] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.384130 32644 tablet_bootstrap.cc:654] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.384363 31391 consensus_queue.cc:1035] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.384696 32641 tablet_bootstrap.cc:492] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:53.385100 32644 tablet_bootstrap.cc:492] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:53.385183 32644 ts_tablet_manager.cc:1367] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:53.385604 32641 tablet_bootstrap.cc:654] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.385625 32644 raft_consensus.cc:348] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.385721 32644 raft_consensus.cc:374] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.385752 32644 raft_consensus.cc:729] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.385862 32644 consensus_queue.cc:260] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.386137 32644 ts_tablet_manager.cc:1398] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:53.387092 27042 tablet_service.cc:1467] Processing CreateTablet for tablet aa8dc7e708a94f0b9947e1b420f1fa40 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.387382 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8dc7e708a94f0b9947e1b420f1fa40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.389577 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet fe68fe7c69554cbeb9984a7be858bb26
I20240502 01:58:53.390071 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet fe68fe7c69554cbeb9984a7be858bb26 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:53 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:53.390101 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet fe68fe7c69554cbeb9984a7be858bb26 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:53 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:53.390313 32627 tablet_replica.cc:330] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:53.390434 32627 raft_consensus.cc:2229] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:53.390460 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc3e48cb92b4788a88c3cbf715d1596 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=731222ffc45743f4a6086df6d34f5405]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:53.390538 32627 raft_consensus.cc:2258] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:53.390676 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc3e48cb92b4788a88c3cbf715d1596. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.390847 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet fe68fe7c69554cbeb9984a7be858bb26 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:53 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:53.391134 32627 ts_tablet_manager.cc:1875] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:53.392707 32627 ts_tablet_manager.cc:1888] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:53.392776 32627 log.cc:1196] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/fe68fe7c69554cbeb9984a7be858bb26
I20240502 01:58:53.392855 32639 tablet_bootstrap.cc:492] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:53.393013 32627 ts_tablet_manager.cc:1909] T fe68fe7c69554cbeb9984a7be858bb26 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:53.393800 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table_new_name [id=34123f56c6cc4db1ba26e43450c24d44]) successfully deleted
I20240502 01:58:53.394143 32639 tablet_bootstrap.cc:654] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.394294 32645 tablet_bootstrap.cc:492] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:53.394884 31391 consensus_queue.cc:1035] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.395452 32640 tablet_bootstrap.cc:492] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:53.396451 32640 tablet_bootstrap.cc:654] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.397089 32652 tablet_replica.cc:330] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:53.397257 32652 raft_consensus.cc:2229] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:53.397002 32639 tablet_bootstrap.cc:492] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:53.397534 32645 tablet_bootstrap.cc:654] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.397635 32652 raft_consensus.cc:2258] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:53.398171 32652 ts_tablet_manager.cc:1875] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:53.398202 32651 tablet_bootstrap.cc:492] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:53.399065 32651 tablet_bootstrap.cc:654] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.397483 32639 ts_tablet_manager.cc:1367] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 01:58:53.400451 32652 ts_tablet_manager.cc:1888] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:53.400694 32639 raft_consensus.cc:348] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.400791 32639 raft_consensus.cc:374] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.400822 32639 raft_consensus.cc:729] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.400952 32639 consensus_queue.cc:260] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.401127 32641 tablet_bootstrap.cc:492] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:53.401214 32641 ts_tablet_manager.cc:1367] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:53.401625 32640 tablet_bootstrap.cc:492] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:53.401597 32641 raft_consensus.cc:348] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.401681 32641 raft_consensus.cc:374] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.401715 32641 raft_consensus.cc:729] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.401826 32641 consensus_queue.cc:260] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.401859 32639 ts_tablet_manager.cc:1398] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:53.401871 32640 ts_tablet_manager.cc:1367] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:53.402091 32641 ts_tablet_manager.cc:1398] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:53.402467 32640 raft_consensus.cc:348] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.402686 32640 raft_consensus.cc:374] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.402770 32640 raft_consensus.cc:729] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.403018 32640 consensus_queue.cc:260] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.403854 32639 tablet_bootstrap.cc:492] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:53.404032 32640 ts_tablet_manager.cc:1398] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:53.404228 32640 tablet_bootstrap.cc:492] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:53.406795 32639 tablet_bootstrap.cc:654] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.408555 32640 tablet_bootstrap.cc:654] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.408864 32639 tablet_bootstrap.cc:492] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:53.408986 32639 ts_tablet_manager.cc:1367] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:53.409186 32653 tablet_replica.cc:330] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:53.409338 32653 raft_consensus.cc:2229] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:53.409462 32653 raft_consensus.cc:2258] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:53.409583 32639 raft_consensus.cc:348] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.409735 32639 raft_consensus.cc:374] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.409811 32639 raft_consensus.cc:729] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.409991 32639 consensus_queue.cc:260] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.410144 32653 ts_tablet_manager.cc:1875] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:53.410172 32643 tablet_bootstrap.cc:492] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:53.410246 32643 ts_tablet_manager.cc:1367] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 01:58:53.410626 32643 raft_consensus.cc:348] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.410712 32643 raft_consensus.cc:374] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.410738 32643 raft_consensus.cc:729] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.410832 32643 consensus_queue.cc:260] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.410948 32639 ts_tablet_manager.cc:1398] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:53.411067 32643 ts_tablet_manager.cc:1398] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:53.411638 32640 tablet_bootstrap.cc:492] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:53.411708 32640 ts_tablet_manager.cc:1367] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:58:53.413184 32640 raft_consensus.cc:348] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.413359 32640 raft_consensus.cc:374] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.413441 32640 raft_consensus.cc:729] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.413600 32640 consensus_queue.cc:260] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.413978 32640 ts_tablet_manager.cc:1398] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:53.403931 31391 consensus_queue.cc:1035] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20240502 01:58:53.400566 32652 log.cc:1196] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/fe68fe7c69554cbeb9984a7be858bb26
I20240502 01:58:53.416597 32652 ts_tablet_manager.cc:1909] T fe68fe7c69554cbeb9984a7be858bb26 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:53.417271 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table_new_name [id=34123f56c6cc4db1ba26e43450c24d44]) successfully deleted
I20240502 01:58:53.418953 32653 ts_tablet_manager.cc:1888] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:53.419075 32653 log.cc:1196] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/fe68fe7c69554cbeb9984a7be858bb26
I20240502 01:58:53.419224 31391 consensus_queue.cc:1035] T 6ac59b967e484772b3366b919757ced7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.419425 32653 ts_tablet_manager.cc:1909] T fe68fe7c69554cbeb9984a7be858bb26 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:53.420616 32651 tablet_bootstrap.cc:492] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:53.420745 32651 ts_tablet_manager.cc:1367] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 01:58:53.421221 32651 raft_consensus.cc:348] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.421308 32651 raft_consensus.cc:374] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.421327 32651 raft_consensus.cc:729] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.421437 32651 consensus_queue.cc:260] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.421746 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet fe68fe7c69554cbeb9984a7be858bb26 (table rebalancer_dynamic_table_new_name [id=34123f56c6cc4db1ba26e43450c24d44]) successfully deleted
I20240502 01:58:53.421911 32651 ts_tablet_manager.cc:1398] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:53.423808 32645 tablet_bootstrap.cc:492] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:53.423902 32645 ts_tablet_manager.cc:1367] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 01:58:53.424312 32645 raft_consensus.cc:348] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.424408 32645 raft_consensus.cc:374] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.424439 32645 raft_consensus.cc:729] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.424544 32645 consensus_queue.cc:260] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.424823 32645 ts_tablet_manager.cc:1398] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:53.448434 29349 raft_consensus.cc:1203] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:53.461112 32661 raft_consensus.cc:482] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:53.461213 32661 raft_consensus.cc:504] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.461567 32661 leader_election.cc:288] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:53.461910 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc3e48cb92b4788a88c3cbf715d1596" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:53.461910 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc3e48cb92b4788a88c3cbf715d1596" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:53.462076 29216 raft_consensus.cc:2454] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:53.462116 26661 raft_consensus.cc:2454] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:53.462441 29285 leader_election.cc:302] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:53.462690 32661 raft_consensus.cc:2781] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:53.462745 32661 raft_consensus.cc:482] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:53.462780 32661 raft_consensus.cc:3037] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.463635 32661 raft_consensus.cc:504] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.463922 32661 leader_election.cc:288] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:53.464217 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc3e48cb92b4788a88c3cbf715d1596" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:53.464287 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc3e48cb92b4788a88c3cbf715d1596" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:53.464345 26661 raft_consensus.cc:3037] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.464385 29216 raft_consensus.cc:3037] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.465225 29216 raft_consensus.cc:2454] T 0dc3e48cb92b4788a88c3cbf715d1596 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:53.465250 26661 raft_consensus.cc:2454] T 0dc3e48cb92b4788a88c3cbf715d1596 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:53.465497 29285 leader_election.cc:302] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:53.465667 32661 raft_consensus.cc:2781] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:53.465775 32661 raft_consensus.cc:686] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:53.465929 32661 consensus_queue.cc:237] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.467553 26509 catalog_manager.cc:5484] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:53.470172 32171 raft_consensus.cc:482] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:53.470256 32171 raft_consensus.cc:504] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.470671 32171 leader_election.cc:288] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:53.470975 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:53.471103 29079 raft_consensus.cc:2454] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:53.471168 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:53.471302 26794 raft_consensus.cc:2454] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:53.471374 29419 leader_election.cc:302] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:53.471530 32171 raft_consensus.cc:2781] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:53.471599 32171 raft_consensus.cc:482] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:53.471639 32171 raft_consensus.cc:3037] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.472404 32171 raft_consensus.cc:504] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.472695 32171 leader_election.cc:288] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:53.472983 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:53.472983 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:53.473096 26794 raft_consensus.cc:3037] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.473096 29079 raft_consensus.cc:3037] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.474013 26794 raft_consensus.cc:2454] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:53.474013 29079 raft_consensus.cc:2454] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:53.474300 29419 leader_election.cc:302] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:53.474464 32171 raft_consensus.cc:2781] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:53.474634 32171 raft_consensus.cc:686] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:53.474757 32171 consensus_queue.cc:237] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:53.476173 26509 catalog_manager.cc:5484] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:53.483811 31991 raft_consensus.cc:482] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:53.483909 31991 raft_consensus.cc:504] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.484252 31991 leader_election.cc:288] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:53.484536 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8dc7e708a94f0b9947e1b420f1fa40" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:53.484655 29349 raft_consensus.cc:2454] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:53.484926 29013 leader_election.cc:302] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:53.484933 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8dc7e708a94f0b9947e1b420f1fa40" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:53.485078 32213 raft_consensus.cc:2781] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:53.485072 27063 raft_consensus.cc:2454] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:53.485121 32213 raft_consensus.cc:482] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:53.485153 32213 raft_consensus.cc:3037] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.485929 32213 raft_consensus.cc:504] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.486312 32213 leader_election.cc:288] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:53.486595 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8dc7e708a94f0b9947e1b420f1fa40" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:53.486712 29349 raft_consensus.cc:3037] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.486814 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8dc7e708a94f0b9947e1b420f1fa40" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:53.486918 27063 raft_consensus.cc:3037] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.487632 29349 raft_consensus.cc:2454] T aa8dc7e708a94f0b9947e1b420f1fa40 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:53.487854 27063 raft_consensus.cc:2454] T aa8dc7e708a94f0b9947e1b420f1fa40 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:53.487898 29013 leader_election.cc:302] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:53.488204 32213 raft_consensus.cc:2781] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:53.488279 32213 raft_consensus.cc:686] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:53.488402 32213 consensus_queue.cc:237] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:53.489830 26509 catalog_manager.cc:5484] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:53.491011 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:53.496615 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]), partition=
I20240502 01:58:53.496805 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]), partition=
I20240502 01:58:53.496861 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4ebdbd1474452e8fa29a36c589c6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.497045 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4ebdbd1474452e8fa29a36c589c6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.499137 32645 tablet_bootstrap.cc:492] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:53.499214 32644 tablet_bootstrap.cc:492] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:53.500010 32645 tablet_bootstrap.cc:654] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.501538 32644 tablet_bootstrap.cc:654] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.502285 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]), partition=
I20240502 01:58:53.502498 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4ebdbd1474452e8fa29a36c589c6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:53.502553 29079 raft_consensus.cc:1203] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:53.502681 26927 raft_consensus.cc:1203] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:53.504534 32640 tablet_bootstrap.cc:492] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:53.505406 32640 tablet_bootstrap.cc:654] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:53.506996 32640 tablet_bootstrap.cc:492] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:53.507078 32640 ts_tablet_manager.cc:1367] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:58:53.507571 32640 raft_consensus.cc:348] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.507670 32640 raft_consensus.cc:374] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.507702 32640 raft_consensus.cc:729] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.507808 32640 consensus_queue.cc:260] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.508092 32640 ts_tablet_manager.cc:1398] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:53.513011 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:53.514359 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:53.515739 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:53.518829 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:53.519225 32644 tablet_bootstrap.cc:492] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:53.519312 32644 ts_tablet_manager.cc:1367] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 01:58:53.519439 32645 tablet_bootstrap.cc:492] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:53.519511 32645 ts_tablet_manager.cc:1367] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 01:58:53.519760 32644 raft_consensus.cc:348] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.519856 32644 raft_consensus.cc:374] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.519886 32644 raft_consensus.cc:729] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.519927 32645 raft_consensus.cc:348] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.520009 32645 raft_consensus.cc:374] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:53.519992 32644 consensus_queue.cc:260] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.520037 32645 raft_consensus.cc:729] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:53.520159 32645 consensus_queue.cc:260] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.520278 32644 ts_tablet_manager.cc:1398] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:53.520423 32645 ts_tablet_manager.cc:1398] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:53.528517 29454 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:53.529228 29079 raft_consensus.cc:1203] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:53.532649 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:53.533252 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:53.543325 29059 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:53.607936 26794 consensus_queue.cc:237] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:53.609019 29216 raft_consensus.cc:1261] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:53.609069 26661 raft_consensus.cc:1261] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:53.609480 32646 consensus_queue.cc:1035] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:53.609617 30910 consensus_queue.cc:1035] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:53.611521 32732 raft_consensus.cc:2932] T 2e129aa347444eafab51845d9bf2ae32 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:53.611994 26658 raft_consensus.cc:2932] T 2e129aa347444eafab51845d9bf2ae32 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:53.611984 29216 raft_consensus.cc:2932] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:53.614835 26509 catalog_manager.cc:5484] T 2e129aa347444eafab51845d9bf2ae32 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:53.617652 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e129aa347444eafab51845d9bf2ae32 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:53.617875 32653 tablet_replica.cc:330] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:53.618014 32653 raft_consensus.cc:2229] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:53.618192 32653 raft_consensus.cc:2258] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:53.618748 32653 ts_tablet_manager.cc:1875] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:53.620757 32653 ts_tablet_manager.cc:1888] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:53.620822 32653 log.cc:1196] T 2e129aa347444eafab51845d9bf2ae32 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/2e129aa347444eafab51845d9bf2ae32
I20240502 01:58:53.621389 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 2e129aa347444eafab51845d9bf2ae32 (table rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]) successfully deleted
I20240502 01:58:53.792438 32669 raft_consensus.cc:482] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:53.792562 32669 raft_consensus.cc:504] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.792949 32669 leader_election.cc:288] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:53.793298 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4ebdbd1474452e8fa29a36c589c6c2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:53.793342 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4ebdbd1474452e8fa29a36c589c6c2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:53.793421 29079 raft_consensus.cc:2454] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:53.793468 29487 raft_consensus.cc:2454] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:53.793704 29147 leader_election.cc:302] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:53.793926 32669 raft_consensus.cc:2781] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:53.793982 32669 raft_consensus.cc:482] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:53.794006 32669 raft_consensus.cc:3037] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.794847 32669 raft_consensus.cc:504] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.795157 32669 leader_election.cc:288] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:53.795444 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4ebdbd1474452e8fa29a36c589c6c2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:53.795557 29487 raft_consensus.cc:3037] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.795545 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4ebdbd1474452e8fa29a36c589c6c2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:53.795640 29079 raft_consensus.cc:3037] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:53.796594 29079 raft_consensus.cc:2454] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:53.796835 29487 raft_consensus.cc:2454] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:53.796859 29147 leader_election.cc:302] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:53.796995 32669 raft_consensus.cc:2781] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:53.797089 32669 raft_consensus.cc:686] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:53.797233 32669 consensus_queue.cc:237] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:53.799396 26509 catalog_manager.cc:5484] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:53.815021 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:53.817746 29454 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:53.821746 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:53.824328 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:53.825603 26909 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:53.827333 29059 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:53.842221 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:53.848845 26774 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:53.901407 29487 consensus_queue.cc:237] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:53.902632 29216 raft_consensus.cc:1261] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:53.902632 26927 raft_consensus.cc:1261] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:53.903035 32697 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:53.903182 32171 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:53.904697 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:53.904930 32171 raft_consensus.cc:2932] T ee632f93fe234584814e6aa7b6a63bc0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:53.906704 29216 raft_consensus.cc:2932] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:53.906894 26927 raft_consensus.cc:2932] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:53.909874 26509 catalog_manager.cc:5484] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:53.912169 29487 raft_consensus.cc:1261] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:53.912405 29079 raft_consensus.cc:1261] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:53.912770 32669 consensus_queue.cc:1035] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.913467 32669 consensus_queue.cc:1035] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:53.915652 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet ee632f93fe234584814e6aa7b6a63bc0 with delete_type TABLET_DATA_TOMBSTONED (TS 468dd27456c64d67b22e8a58af93d93d not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:53.915808 26509 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]). Expected version 1 got 0
I20240502 01:58:53.916205 348 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:53.916519 348 tablet.cc:1620] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:53.917153 26509 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]). Expected version 1 got 0
I20240502 01:58:53.919142 26509 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]). Expected version 1 got 0
I20240502 01:58:53.922420 26507 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4]). Expected version 1 got 0
I20240502 01:58:53.922537 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4] alter complete (version 1)
I20240502 01:58:53.926270 349 tablet_replica.cc:330] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:53.926407 349 raft_consensus.cc:2229] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:53.926523 349 raft_consensus.cc:2258] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:53.926868 349 ts_tablet_manager.cc:1875] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:53.928812 349 ts_tablet_manager.cc:1888] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:53.928874 349 log.cc:1196] T ee632f93fe234584814e6aa7b6a63bc0 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/ee632f93fe234584814e6aa7b6a63bc0
I20240502 01:58:53.929370 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet ee632f93fe234584814e6aa7b6a63bc0 (table rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]) successfully deleted
I20240502 01:58:53.929355 351 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:53.929618 351 tablet.cc:1620] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:53.930464 350 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:53.930755 350 tablet.cc:1620] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:53.997488 338 consensus_queue.cc:1035] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.001528 32697 consensus_queue.cc:1035] T 59c5307ea58d40dc80079b45f17db1d2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.001873 32364 consensus_queue.cc:1035] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.005694 32364 consensus_queue.cc:1035] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.012111 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:54.025430 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 81cd999f76e846bc944a19f7192883be (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.025589 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 3dfd0b3052e04f549308e8aeb4a79858 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.025702 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd999f76e846bc944a19f7192883be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.025805 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dfd0b3052e04f549308e8aeb4a79858. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.028081 410 tablet_bootstrap.cc:492] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:54.028090 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 3dfd0b3052e04f549308e8aeb4a79858 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.028244 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:54.028343 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dfd0b3052e04f549308e8aeb4a79858. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.028908 410 tablet_bootstrap.cc:654] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.030867 413 tablet_bootstrap.cc:492] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:54.030838 26907 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5f8bfeddd1443da789f67949830f99 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.031085 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5f8bfeddd1443da789f67949830f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.031404 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 3dfd0b3052e04f549308e8aeb4a79858 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.031618 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dfd0b3052e04f549308e8aeb4a79858. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.032191 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 81cd999f76e846bc944a19f7192883be (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.032397 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd999f76e846bc944a19f7192883be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.032456 413 tablet_bootstrap.cc:654] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.025709 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5f8bfeddd1443da789f67949830f99 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.033449 413 tablet_bootstrap.cc:492] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:54.033543 413 ts_tablet_manager.cc:1367] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:54.033604 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5f8bfeddd1443da789f67949830f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.034102 415 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:54.034385 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5f8bfeddd1443da789f67949830f99 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.034618 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5f8bfeddd1443da789f67949830f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.033980 413 raft_consensus.cc:348] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.034668 413 raft_consensus.cc:374] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.034709 413 raft_consensus.cc:729] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.034819 413 consensus_queue.cc:260] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.034936 415 tablet_bootstrap.cc:654] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.035095 413 ts_tablet_manager.cc:1398] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:54.036875 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 81cd999f76e846bc944a19f7192883be (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.037094 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd999f76e846bc944a19f7192883be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.038581 29196 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:54.040066 410 tablet_bootstrap.cc:492] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:54.040153 410 ts_tablet_manager.cc:1367] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:58:54.040647 410 raft_consensus.cc:348] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.040745 410 raft_consensus.cc:374] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.040776 410 raft_consensus.cc:729] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.040884 410 consensus_queue.cc:260] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.041139 410 ts_tablet_manager.cc:1398] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.041297 410 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:54.042537 26908 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:54.043442 32213 consensus_queue.cc:1035] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.045989 348 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:54.046296 348 tablet.cc:1620] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:54.046545 417 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:54.046916 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:54.047113 411 tablet_bootstrap.cc:492] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:54.047498 350 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:54.047746 350 tablet.cc:1620] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:54.050413 411 tablet_bootstrap.cc:654] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.048440 417 tablet_bootstrap.cc:654] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.052210 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=10a649e08bb24a25ae793b41ee85c6b4] alter complete (version 2)
I20240502 01:58:54.052506 417 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:54.052572 417 ts_tablet_manager.cc:1367] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:58:54.052985 417 raft_consensus.cc:348] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.053083 417 raft_consensus.cc:374] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.053117 417 raft_consensus.cc:729] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.053220 417 consensus_queue.cc:260] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.053458 417 ts_tablet_manager.cc:1398] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:54.053997 417 tablet_bootstrap.cc:492] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:54.054275 415 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:54.054311 411 tablet_bootstrap.cc:492] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:54.054383 415 ts_tablet_manager.cc:1367] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 01:58:54.054426 411 ts_tablet_manager.cc:1367] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:58:54.054842 415 raft_consensus.cc:348] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.055002 415 raft_consensus.cc:374] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.054997 411 raft_consensus.cc:348] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.055085 415 raft_consensus.cc:729] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.055197 411 raft_consensus.cc:374] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.055562 411 raft_consensus.cc:729] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.055743 411 consensus_queue.cc:260] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.055233 415 consensus_queue.cc:260] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.057394 415 ts_tablet_manager.cc:1398] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:58:54.057614 415 tablet_bootstrap.cc:492] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:54.057859 410 tablet_bootstrap.cc:654] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.058576 415 tablet_bootstrap.cc:654] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.058566 411 ts_tablet_manager.cc:1398] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:54.059646 410 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:54.059746 410 ts_tablet_manager.cc:1367] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20240502 01:58:54.060228 410 raft_consensus.cc:348] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.060384 410 raft_consensus.cc:374] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.060458 410 raft_consensus.cc:729] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.060639 410 consensus_queue.cc:260] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.061051 410 ts_tablet_manager.cc:1398] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:54.061604 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.061798 415 tablet_bootstrap.cc:492] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:54.061878 415 ts_tablet_manager.cc:1367] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:54.062295 415 raft_consensus.cc:348] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.062388 415 raft_consensus.cc:374] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.062422 415 raft_consensus.cc:729] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.062531 415 consensus_queue.cc:260] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.062794 415 ts_tablet_manager.cc:1398] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.062950 415 tablet_bootstrap.cc:492] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:54.063761 415 tablet_bootstrap.cc:654] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.065541 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:54.067438 351 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:54.067695 351 tablet.cc:1620] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:54.067734 417 tablet_bootstrap.cc:654] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.072535 29059 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:54.075186 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:54.077059 29451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:58:54.079931 415 tablet_bootstrap.cc:492] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:54.080008 415 ts_tablet_manager.cc:1367] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 01:58:54.080389 415 raft_consensus.cc:348] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.080487 415 raft_consensus.cc:374] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.080519 415 raft_consensus.cc:729] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.080621 415 consensus_queue.cc:260] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.080904 415 ts_tablet_manager.cc:1398] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:54.082713 32213 consensus_queue.cc:1035] T aa8dc7e708a94f0b9947e1b420f1fa40 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.086099 32697 raft_consensus.cc:482] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:54.086179 32697 raft_consensus.cc:504] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.086524 32697 leader_election.cc:288] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:54.086895 390 raft_consensus.cc:482] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:54.086972 390 raft_consensus.cc:504] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.087301 390 leader_election.cc:288] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:54.087647 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfd0b3052e04f549308e8aeb4a79858" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:54.087705 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfd0b3052e04f549308e8aeb4a79858" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:54.087771 29216 raft_consensus.cc:2454] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:54.087803 26793 raft_consensus.cc:2454] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:58:54.087807 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:54.087941 26794 raft_consensus.cc:2454] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:54.088140 26861 leader_election.cc:302] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:54.088436 29421 leader_election.cc:302] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:54.088945 32697 raft_consensus.cc:2781] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:54.088995 32697 raft_consensus.cc:482] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:54.089030 32697 raft_consensus.cc:3037] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.090332 390 raft_consensus.cc:2781] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:54.090389 390 raft_consensus.cc:482] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:54.090421 390 raft_consensus.cc:3037] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.091311 390 raft_consensus.cc:504] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.091614 390 leader_election.cc:288] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:54.092872 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:54.092973 26927 raft_consensus.cc:2454] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:54.093384 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfd0b3052e04f549308e8aeb4a79858" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:54.093477 26794 raft_consensus.cc:3037] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.094399 26794 raft_consensus.cc:2454] T 3dfd0b3052e04f549308e8aeb4a79858 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:54.094754 26861 leader_election.cc:302] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:54.095110 319 raft_consensus.cc:2781] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:54.095188 319 raft_consensus.cc:686] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:58:54.095299 319 consensus_queue.cc:237] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.089990 32697 raft_consensus.cc:504] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.097105 32697 leader_election.cc:288] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:54.097422 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:54.097512 26794 raft_consensus.cc:3037] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.097566 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:54.097681 26927 raft_consensus.cc:3037] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.098351 26794 raft_consensus.cc:2454] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:54.098541 26927 raft_consensus.cc:2454] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:54.098608 29421 leader_election.cc:302] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:54.098744 32697 raft_consensus.cc:2781] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:54.098807 32697 raft_consensus.cc:686] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:54.098909 32697 consensus_queue.cc:237] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.099372 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dfd0b3052e04f549308e8aeb4a79858" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:54.099476 29216 raft_consensus.cc:3037] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.099620 26507 catalog_manager.cc:5484] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:54.100389 29216 raft_consensus.cc:2454] T 3dfd0b3052e04f549308e8aeb4a79858 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:58:54.102392 26510 catalog_manager.cc:5484] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:54.118702 417 tablet_bootstrap.cc:492] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:54.118784 417 ts_tablet_manager.cc:1367] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.000s
I20240502 01:58:54.119230 417 raft_consensus.cc:348] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.119324 417 raft_consensus.cc:374] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.119350 417 raft_consensus.cc:729] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.119452 417 consensus_queue.cc:260] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.119699 417 ts_tablet_manager.cc:1398] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:54.143963 26658 consensus_queue.cc:237] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.145112 29079 raft_consensus.cc:1261] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:54.145206 26927 raft_consensus.cc:1261] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:54.145648 447 consensus_queue.cc:1035] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:54.146312 447 consensus_queue.cc:1035] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:54.148358 31766 raft_consensus.cc:2932] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:54.148836 26927 raft_consensus.cc:2932] T 99da1c4053674589b87a7898ed9d57de P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:54.152396 26507 catalog_manager.cc:5484] T 99da1c4053674589b87a7898ed9d57de P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:54.155287 29079 raft_consensus.cc:2932] T 99da1c4053674589b87a7898ed9d57de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:54.157927 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 99da1c4053674589b87a7898ed9d57de with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:54.158479 455 tablet_replica.cc:330] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:54.158629 455 raft_consensus.cc:2229] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:54.158780 455 raft_consensus.cc:2258] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.159404 455 ts_tablet_manager.cc:1875] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:54.160050 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:54.163147 455 ts_tablet_manager.cc:1888] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:54.163205 455 log.cc:1196] T 99da1c4053674589b87a7898ed9d57de P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/99da1c4053674589b87a7898ed9d57de
I20240502 01:58:54.163637 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 99da1c4053674589b87a7898ed9d57de (table rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]) successfully deleted
I20240502 01:58:54.166301 348 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:54.166695 351 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:54.170462 350 tablet.cc:1722] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:54.172461 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=10a649e08bb24a25ae793b41ee85c6b4] alter complete (version 3)
I20240502 01:58:54.276661 26512 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:54.276787 26512 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:54.278649 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 8e4ebdbd1474452e8fa29a36c589c6c2
I20240502 01:58:54.279296 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:54 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:54.279595 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:54 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:54.279635 494 tablet_replica.cc:330] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:54.279768 494 raft_consensus.cc:2229] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:54.279851 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4ebdbd1474452e8fa29a36c589c6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:54 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:54.279894 494 raft_consensus.cc:2258] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.280172 495 tablet_replica.cc:330] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:54.280310 495 raft_consensus.cc:2229] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:54.280324 496 tablet_replica.cc:330] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:54.280421 495 raft_consensus.cc:2258] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.280444 496 raft_consensus.cc:2229] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:54.280489 494 ts_tablet_manager.cc:1875] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:54.280674 496 raft_consensus.cc:2258] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.280920 495 ts_tablet_manager.cc:1875] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:54.281215 496 ts_tablet_manager.cc:1875] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:54.282708 494 ts_tablet_manager.cc:1888] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:54.282784 494 log.cc:1196] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/8e4ebdbd1474452e8fa29a36c589c6c2
I20240502 01:58:54.283066 494 ts_tablet_manager.cc:1909] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:54.283583 495 ts_tablet_manager.cc:1888] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:54.283660 495 log.cc:1196] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/8e4ebdbd1474452e8fa29a36c589c6c2
I20240502 01:58:54.283905 495 ts_tablet_manager.cc:1909] T 8e4ebdbd1474452e8fa29a36c589c6c2 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:54.284488 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table_new_name [id=10a649e08bb24a25ae793b41ee85c6b4]) successfully deleted
I20240502 01:58:54.288483 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table_new_name [id=10a649e08bb24a25ae793b41ee85c6b4]) successfully deleted
I20240502 01:58:54.289016 496 ts_tablet_manager.cc:1888] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:54.289117 496 log.cc:1196] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/8e4ebdbd1474452e8fa29a36c589c6c2
I20240502 01:58:54.289358 496 ts_tablet_manager.cc:1909] T 8e4ebdbd1474452e8fa29a36c589c6c2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:54.289932 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 8e4ebdbd1474452e8fa29a36c589c6c2 (table rebalancer_dynamic_table_new_name [id=10a649e08bb24a25ae793b41ee85c6b4]) successfully deleted
I20240502 01:58:54.324900 32213 raft_consensus.cc:482] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:54.324990 32213 raft_consensus.cc:504] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.325381 32213 leader_election.cc:288] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:54.325655 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd999f76e846bc944a19f7192883be" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:54.325738 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd999f76e846bc944a19f7192883be" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:54.325801 26927 raft_consensus.cc:2454] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:54.325855 29487 raft_consensus.cc:2454] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:54.326118 29016 leader_election.cc:302] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:54.326277 32213 raft_consensus.cc:2781] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:54.326344 32213 raft_consensus.cc:482] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:54.326386 32213 raft_consensus.cc:3037] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.327070 32213 raft_consensus.cc:504] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.327399 32213 leader_election.cc:288] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:54.327695 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd999f76e846bc944a19f7192883be" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:54.327791 26927 raft_consensus.cc:3037] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.327759 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd999f76e846bc944a19f7192883be" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:54.327862 29487 raft_consensus.cc:3037] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.328536 26927 raft_consensus.cc:2454] T 81cd999f76e846bc944a19f7192883be P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:54.328675 29487 raft_consensus.cc:2454] T 81cd999f76e846bc944a19f7192883be P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:54.328934 29016 leader_election.cc:302] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:54.329089 32213 raft_consensus.cc:2781] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:54.329169 32213 raft_consensus.cc:686] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:54.329291 32213 consensus_queue.cc:237] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.330993 26512 catalog_manager.cc:5484] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:54.345366 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:54.353987 29059 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.356770 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:54.357503 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.359777 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:54.366748 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:54.374217 29451 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:54.377725 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:54.386310 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:54.392313 29059 tablet_service.cc:1467] Processing CreateTablet for tablet b389e5cc088447eb90bd99b29a2a53b5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2]), partition=
I20240502 01:58:54.392586 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b389e5cc088447eb90bd99b29a2a53b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.392719 26775 tablet_service.cc:1467] Processing CreateTablet for tablet b389e5cc088447eb90bd99b29a2a53b5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2]), partition=
I20240502 01:58:54.392932 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b389e5cc088447eb90bd99b29a2a53b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.395327 410 tablet_bootstrap.cc:492] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:54.395356 29195 tablet_service.cc:1467] Processing CreateTablet for tablet b389e5cc088447eb90bd99b29a2a53b5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2]), partition=
I20240502 01:58:54.395565 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b389e5cc088447eb90bd99b29a2a53b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.396203 410 tablet_bootstrap.cc:654] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.397500 413 tablet_bootstrap.cc:492] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:54.397605 411 tablet_bootstrap.cc:492] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:54.398381 413 tablet_bootstrap.cc:654] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.398381 411 tablet_bootstrap.cc:654] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.402889 410 tablet_bootstrap.cc:492] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:54.402982 410 ts_tablet_manager.cc:1367] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240502 01:58:54.403376 410 raft_consensus.cc:348] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.403481 410 raft_consensus.cc:374] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.403520 410 raft_consensus.cc:729] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.403631 410 consensus_queue.cc:260] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.403861 411 tablet_bootstrap.cc:492] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:54.403898 410 ts_tablet_manager.cc:1398] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:54.403949 411 ts_tablet_manager.cc:1367] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 01:58:54.404331 411 raft_consensus.cc:348] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.404428 411 raft_consensus.cc:374] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.404453 411 raft_consensus.cc:729] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.404546 411 consensus_queue.cc:260] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.404812 411 ts_tablet_manager.cc:1398] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:54.409426 413 tablet_bootstrap.cc:492] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:54.409511 413 ts_tablet_manager.cc:1367] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:58:54.409925 413 raft_consensus.cc:348] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.410048 413 raft_consensus.cc:374] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.410077 413 raft_consensus.cc:729] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.410181 413 consensus_queue.cc:260] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.410431 413 ts_tablet_manager.cc:1398] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:54.444383 29079 consensus_queue.cc:237] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:54.445489 26794 raft_consensus.cc:1261] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:54.445550 29349 raft_consensus.cc:1261] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:54.445940 425 consensus_queue.cc:1035] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:54.446187 32213 consensus_queue.cc:1035] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:54.447850 425 raft_consensus.cc:2932] T 6acba26a44294eb284d03176631ae39b P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:54.448204 26794 raft_consensus.cc:2932] T 6acba26a44294eb284d03176631ae39b P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:54.448575 29349 raft_consensus.cc:2932] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:54.451287 26515 catalog_manager.cc:5484] T 6acba26a44294eb284d03176631ae39b P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:54.453795 27042 tablet_service.cc:1514] Processing DeleteTablet for tablet 6acba26a44294eb284d03176631ae39b with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:54.453997 455 tablet_replica.cc:330] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:54.454150 455 raft_consensus.cc:2229] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:54.454259 455 raft_consensus.cc:2258] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.454644 455 ts_tablet_manager.cc:1875] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:54.456624 455 ts_tablet_manager.cc:1888] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:54.456673 455 log.cc:1196] T 6acba26a44294eb284d03176631ae39b P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/6acba26a44294eb284d03176631ae39b
I20240502 01:58:54.458550 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 6acba26a44294eb284d03176631ae39b (table rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]) successfully deleted
I20240502 01:58:54.519817 390 consensus_queue.cc:1035] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.521924 339 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.523090 319 consensus_queue.cc:1035] T 3dfd0b3052e04f549308e8aeb4a79858 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.525674 32697 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.527583 32669 raft_consensus.cc:482] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:54.527669 32669 raft_consensus.cc:504] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.528048 32669 leader_election.cc:288] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:54.528349 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:54.528375 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:54.528462 26794 raft_consensus.cc:2454] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:54.528442 529 raft_consensus.cc:482] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:54.528532 529 raft_consensus.cc:504] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.528748 29149 leader_election.cc:302] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:54.528913 32669 raft_consensus.cc:2781] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:54.528913 529 leader_election.cc:288] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:54.528966 32669 raft_consensus.cc:482] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:54.529011 32669 raft_consensus.cc:3037] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.529158 29079 raft_consensus.cc:2454] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:54.529206 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:54.529233 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:54.529336 26794 raft_consensus.cc:2454] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:54.529569 29015 leader_election.cc:302] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:54.529763 529 raft_consensus.cc:2781] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:54.529819 529 raft_consensus.cc:482] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:54.529855 529 raft_consensus.cc:3037] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.529969 32669 raft_consensus.cc:504] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.530203 29216 raft_consensus.cc:2379] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3d18a019013487ab801599cfb80e4b4 in current term 1: Already voted for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in this term.
I20240502 01:58:54.530323 32669 leader_election.cc:288] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:54.530568 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:54.530668 26794 raft_consensus.cc:3037] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:54.530874 529 raft_consensus.cc:504] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.531172 529 leader_election.cc:288] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:54.531564 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:54.531576 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:54.531625 26794 raft_consensus.cc:2454] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:54.531684 29216 raft_consensus.cc:2379] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3d18a019013487ab801599cfb80e4b4 in current term 1: Already voted for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in this term.
I20240502 01:58:54.531903 29149 leader_election.cc:302] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:54.532094 32669 raft_consensus.cc:2781] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:54.532176 32669 raft_consensus.cc:686] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:54.532163 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b389e5cc088447eb90bd99b29a2a53b5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:54.532326 29079 raft_consensus.cc:2379] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in current term 1: Already voted for candidate f3d18a019013487ab801599cfb80e4b4 in this term.
I20240502 01:58:54.532299 32669 consensus_queue.cc:237] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:54.532902 29015 leader_election.cc:302] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d18a019013487ab801599cfb80e4b4; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124
I20240502 01:58:54.533740 529 raft_consensus.cc:2726] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 01:58:54.534250 26512 catalog_manager.cc:5484] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:54.552062 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:54.555001 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:54.563869 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:54.568957 29059 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.571890 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:54.573942 29451 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:54.580847 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.614063 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:54.649660 29079 consensus_queue.cc:237] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:54.650899 29350 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:54.650959 27063 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:54.651278 425 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:54.651479 529 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:54.652168 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:54.653215 529 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.654194 27063 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.654224 29350 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.657083 26507 catalog_manager.cc:5484] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.657297 29079 raft_consensus.cc:1261] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:54.657724 462 consensus_queue.cc:1035] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.657897 26794 raft_consensus.cc:1261] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:54.658701 462 consensus_queue.cc:1035] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.661109 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet 8afd4c5362464c4cba43d5ef28b50c29 with delete_type TABLET_DATA_TOMBSTONED (TS 7c693d0933c64d9babc07f404df45cd1 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:54.661350 494 tablet_replica.cc:330] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:54.661399 26512 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet b389e5cc088447eb90bd99b29a2a53b5 (table rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2]). Expected version 1 got 0
I20240502 01:58:54.661509 494 raft_consensus.cc:2229] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:54.661633 494 raft_consensus.cc:2258] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.662093 494 ts_tablet_manager.cc:1875] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:54.662576 26509 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet b389e5cc088447eb90bd99b29a2a53b5 (table rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2]). Expected version 1 got 0
I20240502 01:58:54.663329 348 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:54.663604 348 tablet.cc:1620] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:54.664254 494 ts_tablet_manager.cc:1888] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:54.664330 494 log.cc:1196] T 8afd4c5362464c4cba43d5ef28b50c29 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/8afd4c5362464c4cba43d5ef28b50c29
I20240502 01:58:54.664641 351 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:54.664952 351 tablet.cc:1620] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:54.666815 616 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:54.667089 616 tablet.cc:1620] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:54.667181 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 8afd4c5362464c4cba43d5ef28b50c29 (table rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]) successfully deleted
I20240502 01:58:54.667915 26507 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet b389e5cc088447eb90bd99b29a2a53b5 (table rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2]). Expected version 1 got 0
I20240502 01:58:54.668123 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2] alter complete (version 1)
I20240502 01:58:54.712193 425 consensus_queue.cc:1035] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.724473 529 consensus_queue.cc:1035] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:54.748191 27042 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:54.750694 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:54.758080 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:54.758502 29451 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:58:54.758607 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:54.762148 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:54.768939 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.774221 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:54.781389 348 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:54.781690 348 tablet.cc:1620] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:54.782066 351 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:54.782335 351 tablet.cc:1620] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:54.786825 616 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:54.786985 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=be4cccea619348d8bcc5bfb9fbfc7bc2] alter complete (version 2)
I20240502 01:58:54.787096 616 tablet.cc:1620] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:54.797088 29059 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:54.841760 26658 consensus_queue.cc:237] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:54.843039 26794 raft_consensus.cc:1261] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:54.843099 29350 raft_consensus.cc:1261] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:54.843401 31766 consensus_queue.cc:1035] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:54.843621 447 consensus_queue.cc:1035] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:54.845299 31766 raft_consensus.cc:2932] T 50bfc86904bc4347bd6beb7f16eb638d P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.845712 29350 raft_consensus.cc:2932] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.848753 26513 catalog_manager.cc:5484] T 50bfc86904bc4347bd6beb7f16eb638d P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.848783 26794 raft_consensus.cc:2932] T 50bfc86904bc4347bd6beb7f16eb638d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:54.851913 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 50bfc86904bc4347bd6beb7f16eb638d with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:54.852242 692 tablet_replica.cc:330] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:54.852409 692 raft_consensus.cc:2229] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:54.852551 692 raft_consensus.cc:2258] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:54.852977 692 ts_tablet_manager.cc:1875] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:54.854776 692 ts_tablet_manager.cc:1888] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:54.854861 692 log.cc:1196] T 50bfc86904bc4347bd6beb7f16eb638d P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/50bfc86904bc4347bd6beb7f16eb638d
I20240502 01:58:54.855304 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 50bfc86904bc4347bd6beb7f16eb638d (table rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]) successfully deleted
I20240502 01:58:54.856098 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:54.862426 29195 tablet_service.cc:1467] Processing CreateTablet for tablet bb13643f981847c4bb480b74a4764c4c (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.862711 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb13643f981847c4bb480b74a4764c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.863188 27043 tablet_service.cc:1467] Processing CreateTablet for tablet c3d890af39194df197564cb91e80297e (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.863188 27042 tablet_service.cc:1467] Processing CreateTablet for tablet bb13643f981847c4bb480b74a4764c4c (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.863493 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3d890af39194df197564cb91e80297e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.863610 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb13643f981847c4bb480b74a4764c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.864431 413 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:54.864903 29330 tablet_service.cc:1467] Processing CreateTablet for tablet e66b19b5551b4011bf1bbb6234a2aba3 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.865103 27044 tablet_service.cc:1467] Processing CreateTablet for tablet e66b19b5551b4011bf1bbb6234a2aba3 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.865159 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e66b19b5551b4011bf1bbb6234a2aba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.865335 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e66b19b5551b4011bf1bbb6234a2aba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.865361 413 tablet_bootstrap.cc:654] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.866139 29454 tablet_service.cc:1467] Processing CreateTablet for tablet e66b19b5551b4011bf1bbb6234a2aba3 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.866393 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e66b19b5551b4011bf1bbb6234a2aba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.866374 29451 tablet_service.cc:1467] Processing CreateTablet for tablet bb13643f981847c4bb480b74a4764c4c (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.866559 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb13643f981847c4bb480b74a4764c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.866802 29468 tablet_service.cc:1467] Processing CreateTablet for tablet c3d890af39194df197564cb91e80297e (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.867000 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3d890af39194df197564cb91e80297e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.868906 699 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:54.869558 413 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:54.869647 413 ts_tablet_manager.cc:1367] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 01:58:54.870123 413 raft_consensus.cc:348] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.870220 413 raft_consensus.cc:374] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.870252 413 raft_consensus.cc:729] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.870296 29195 tablet_service.cc:1467] Processing CreateTablet for tablet c3d890af39194df197564cb91e80297e (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ca9eb6b784dd44329e3443cd4a725220]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:54.870371 413 consensus_queue.cc:260] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.870517 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3d890af39194df197564cb91e80297e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:54.870636 413 ts_tablet_manager.cc:1398] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.871353 699 tablet_bootstrap.cc:654] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.871702 701 tablet_bootstrap.cc:492] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:54.872411 701 tablet_bootstrap.cc:654] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.873258 413 tablet_bootstrap.cc:492] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:54.873442 702 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:54.874233 413 tablet_bootstrap.cc:654] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.874351 702 tablet_bootstrap.cc:654] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.874692 699 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:54.874768 699 ts_tablet_manager.cc:1367] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:54.875191 699 raft_consensus.cc:348] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.875311 699 raft_consensus.cc:374] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.875340 413 tablet_bootstrap.cc:492] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:54.875344 699 raft_consensus.cc:729] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.875411 413 ts_tablet_manager.cc:1367] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:54.875458 702 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:54.875455 699 consensus_queue.cc:260] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.875522 702 ts_tablet_manager.cc:1367] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:54.875718 699 ts_tablet_manager.cc:1398] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.875875 699 tablet_bootstrap.cc:492] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:54.875826 413 raft_consensus.cc:348] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.875913 413 raft_consensus.cc:374] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.875877 702 raft_consensus.cc:348] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.875945 413 raft_consensus.cc:729] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.875959 702 raft_consensus.cc:374] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.875985 702 raft_consensus.cc:729] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.876053 413 consensus_queue.cc:260] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.876145 702 consensus_queue.cc:260] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.876334 413 ts_tablet_manager.cc:1398] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:54.876700 699 tablet_bootstrap.cc:654] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.876856 702 ts_tablet_manager.cc:1398] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.876988 702 tablet_bootstrap.cc:492] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:54.879774 702 tablet_bootstrap.cc:654] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.880777 699 tablet_bootstrap.cc:492] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:54.880873 699 ts_tablet_manager.cc:1367] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:54.881270 699 raft_consensus.cc:348] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.881374 699 raft_consensus.cc:374] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.881448 699 raft_consensus.cc:729] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.881573 699 consensus_queue.cc:260] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.881899 699 ts_tablet_manager.cc:1398] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.882084 699 tablet_bootstrap.cc:492] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:54.882932 699 tablet_bootstrap.cc:654] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.884280 699 tablet_bootstrap.cc:492] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:54.884356 699 ts_tablet_manager.cc:1367] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:54.884752 699 raft_consensus.cc:348] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:54.884838 699 raft_consensus.cc:374] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.884868 699 raft_consensus.cc:729] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.884972 699 consensus_queue.cc:260] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:54.885221 699 ts_tablet_manager.cc:1398] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:54.885283 701 tablet_bootstrap.cc:492] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:54.885351 701 ts_tablet_manager.cc:1367] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240502 01:58:54.885696 701 raft_consensus.cc:348] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:54.885807 701 raft_consensus.cc:374] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.885840 701 raft_consensus.cc:729] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.886245 701 consensus_queue.cc:260] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:54.886564 701 ts_tablet_manager.cc:1398] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.887857 702 tablet_bootstrap.cc:492] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:54.887971 702 ts_tablet_manager.cc:1367] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:58:54.888520 702 raft_consensus.cc:348] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:54.888639 702 raft_consensus.cc:374] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.888682 702 raft_consensus.cc:729] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.888813 702 consensus_queue.cc:260] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:54.889472 702 ts_tablet_manager.cc:1398] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.890198 702 tablet_bootstrap.cc:492] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:54.891083 702 tablet_bootstrap.cc:654] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:54.891923 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:54.892318 702 tablet_bootstrap.cc:492] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:54.892390 702 ts_tablet_manager.cc:1367] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:54.893065 702 raft_consensus.cc:348] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.893162 702 raft_consensus.cc:374] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:54.893193 702 raft_consensus.cc:729] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:54.893293 702 consensus_queue.cc:260] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:54.893558 702 ts_tablet_manager.cc:1398] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:54.899113 348 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:54.899338 351 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:54.902959 616 tablet.cc:1722] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:54.904069 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=be4cccea619348d8bcc5bfb9fbfc7bc2] alter complete (version 3)
I20240502 01:58:55.006129 32697 raft_consensus.cc:482] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.006247 32697 raft_consensus.cc:504] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.006740 32697 leader_election.cc:288] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:55.006995 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:55.007131 29216 raft_consensus.cc:2454] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.007186 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:55.007180 26509 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:55.007292 26509 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:55.007310 27063 raft_consensus.cc:2454] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.007500 29421 leader_election.cc:302] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.007902 32697 raft_consensus.cc:2781] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.007961 32697 raft_consensus.cc:482] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.007995 32697 raft_consensus.cc:3037] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.009088 32697 raft_consensus.cc:504] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.009433 32697 leader_election.cc:288] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:55.009774 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:55.009868 26509 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet b389e5cc088447eb90bd99b29a2a53b5
I20240502 01:58:55.009882 27063 raft_consensus.cc:3037] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.010219 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:55.010298 32697 raft_consensus.cc:482] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.010318 29216 raft_consensus.cc:3037] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.010370 32697 raft_consensus.cc:504] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.010664 32697 leader_election.cc:288] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:55.010897 27063 raft_consensus.cc:2454] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.011178 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:55.011238 29216 raft_consensus.cc:2454] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.011281 29422 leader_election.cc:302] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.011284 29213 raft_consensus.cc:2454] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.011372 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet b389e5cc088447eb90bd99b29a2a53b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:55 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:55.011463 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet b389e5cc088447eb90bd99b29a2a53b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:55 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:55.011616 29421 leader_election.cc:302] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.011605 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:55.011724 27063 raft_consensus.cc:2454] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.011718 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet b389e5cc088447eb90bd99b29a2a53b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:55 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:55.011766 339 raft_consensus.cc:2781] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.011905 339 raft_consensus.cc:686] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:55.012037 742 tablet_replica.cc:330] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:55.012037 339 consensus_queue.cc:237] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.012127 32697 raft_consensus.cc:2781] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.012172 32697 raft_consensus.cc:482] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.012199 742 raft_consensus.cc:2229] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.012216 32697 raft_consensus.cc:3037] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.012320 742 raft_consensus.cc:2258] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.012900 742 ts_tablet_manager.cc:1875] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:55.012956 741 tablet_replica.cc:330] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:55.013063 741 raft_consensus.cc:2229] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:55.013368 741 raft_consensus.cc:2258] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.013634 743 tablet_replica.cc:330] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:55.013831 743 raft_consensus.cc:2229] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.013980 743 raft_consensus.cc:2258] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.013955 32697 raft_consensus.cc:504] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.013900 26509 catalog_manager.cc:5484] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.014590 743 ts_tablet_manager.cc:1875] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:55.014600 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:55.014686 27063 raft_consensus.cc:3037] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.015202 742 ts_tablet_manager.cc:1888] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:55.015275 742 log.cc:1196] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/b389e5cc088447eb90bd99b29a2a53b5
I20240502 01:58:55.015484 27063 raft_consensus.cc:2454] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.015527 742 ts_tablet_manager.cc:1909] T b389e5cc088447eb90bd99b29a2a53b5 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:55.016011 29422 leader_election.cc:302] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.016103 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet b389e5cc088447eb90bd99b29a2a53b5 (table rebalancer_dynamic_table_new_name [id=be4cccea619348d8bcc5bfb9fbfc7bc2]) successfully deleted
I20240502 01:58:55.016191 744 raft_consensus.cc:2781] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.016266 744 raft_consensus.cc:686] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:55.016374 744 consensus_queue.cc:237] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.016541 741 ts_tablet_manager.cc:1875] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:55.017401 743 ts_tablet_manager.cc:1888] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:55.017498 743 log.cc:1196] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/b389e5cc088447eb90bd99b29a2a53b5
I20240502 01:58:55.017975 743 ts_tablet_manager.cc:1909] T b389e5cc088447eb90bd99b29a2a53b5 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:55.018368 26509 catalog_manager.cc:5484] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.018707 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet b389e5cc088447eb90bd99b29a2a53b5 (table rebalancer_dynamic_table_new_name [id=be4cccea619348d8bcc5bfb9fbfc7bc2]) successfully deleted
I20240502 01:58:55.018719 741 ts_tablet_manager.cc:1888] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:55.018795 741 log.cc:1196] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/b389e5cc088447eb90bd99b29a2a53b5
I20240502 01:58:55.018911 32697 leader_election.cc:288] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:55.019055 741 ts_tablet_manager.cc:1909] T b389e5cc088447eb90bd99b29a2a53b5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:55.019487 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:55.019590 29213 raft_consensus.cc:3037] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.019693 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet b389e5cc088447eb90bd99b29a2a53b5 (table rebalancer_dynamic_table_new_name [id=be4cccea619348d8bcc5bfb9fbfc7bc2]) successfully deleted
I20240502 01:58:55.020391 29213 raft_consensus.cc:2454] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.111606 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:55.116369 29330 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5b10edbb943509186ea012abdcaf4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]), partition=
I20240502 01:58:55.116446 26908 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5b10edbb943509186ea012abdcaf4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]), partition=
I20240502 01:58:55.116633 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5b10edbb943509186ea012abdcaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.116632 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5b10edbb943509186ea012abdcaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.118628 29468 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5b10edbb943509186ea012abdcaf4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]), partition=
I20240502 01:58:55.118710 701 tablet_bootstrap.cc:492] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:55.118736 32697 raft_consensus.cc:482] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.118841 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5b10edbb943509186ea012abdcaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.118818 32697 raft_consensus.cc:504] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:55.118978 748 tablet_bootstrap.cc:492] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:55.119171 32697 leader_election.cc:288] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:55.119622 701 tablet_bootstrap.cc:654] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.119774 748 tablet_bootstrap.cc:654] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.119902 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:55.120018 27063 raft_consensus.cc:2454] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.120492 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:55.120558 29422 leader_election.cc:302] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.120608 29350 raft_consensus.cc:2454] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.120703 32697 raft_consensus.cc:2781] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.120743 32697 raft_consensus.cc:482] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.120775 32697 raft_consensus.cc:3037] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.121563 32697 raft_consensus.cc:504] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:55.121866 32697 leader_election.cc:288] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:55.122012 702 tablet_bootstrap.cc:492] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:55.122568 748 tablet_bootstrap.cc:492] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:55.122638 748 ts_tablet_manager.cc:1367] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:55.122910 702 tablet_bootstrap.cc:654] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.123093 748 raft_consensus.cc:348] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.123186 748 raft_consensus.cc:374] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.123215 748 raft_consensus.cc:729] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.123333 748 consensus_queue.cc:260] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.123603 748 ts_tablet_manager.cc:1398] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:55.124109 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:55.124236 29350 raft_consensus.cc:3037] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.124320 702 tablet_bootstrap.cc:492] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:55.124392 702 ts_tablet_manager.cc:1367] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:58:55.124794 702 raft_consensus.cc:348] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.124882 702 raft_consensus.cc:374] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.124904 702 raft_consensus.cc:729] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.125000 702 consensus_queue.cc:260] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.125074 29350 raft_consensus.cc:2454] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.125264 702 ts_tablet_manager.cc:1398] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:55.126360 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:55.126469 27063 raft_consensus.cc:3037] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.126463 29419 leader_election.cc:302] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.126797 32697 raft_consensus.cc:2781] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.126873 32697 raft_consensus.cc:686] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:55.126971 32697 consensus_queue.cc:237] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:55.127364 27063 raft_consensus.cc:2454] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.128633 701 tablet_bootstrap.cc:492] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:55.128729 701 ts_tablet_manager.cc:1367] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 01:58:55.129122 701 raft_consensus.cc:348] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.129215 701 raft_consensus.cc:374] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.129242 701 raft_consensus.cc:729] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.129339 701 consensus_queue.cc:260] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.129566 701 ts_tablet_manager.cc:1398] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:55.140105 26509 catalog_manager.cc:5484] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:55.162288 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.165575 29330 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 01:58:55.167343 29468 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.167832 27044 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:55.174269 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:55.181869 29059 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:55.190711 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.198163 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:55.210705 32364 raft_consensus.cc:482] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.210795 32364 raft_consensus.cc:504] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.211197 32364 leader_election.cc:288] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:55.211993 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5b10edbb943509186ea012abdcaf4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:55.212138 26927 raft_consensus.cc:2454] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:55.212471 29286 leader_election.cc:302] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:55.212664 32364 raft_consensus.cc:2781] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.212713 32364 raft_consensus.cc:482] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.212749 32364 raft_consensus.cc:3037] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.213730 32364 raft_consensus.cc:504] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.214044 32364 leader_election.cc:288] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:55.214558 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5b10edbb943509186ea012abdcaf4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:55.214641 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5b10edbb943509186ea012abdcaf4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:55.214679 29489 raft_consensus.cc:3037] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.214732 26927 raft_consensus.cc:3037] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.215596 26927 raft_consensus.cc:2454] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:55.215595 29489 raft_consensus.cc:2454] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:55.216779 29286 leader_election.cc:302] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:55.217108 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5b10edbb943509186ea012abdcaf4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:55.217231 29487 raft_consensus.cc:2362] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1. Re-sending same reply.
I20240502 01:58:55.217618 32364 raft_consensus.cc:2781] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.217777 32364 raft_consensus.cc:686] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:55.217905 32364 consensus_queue.cc:237] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.219679 26509 catalog_manager.cc:5484] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.245785 26927 consensus_queue.cc:237] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:55.246925 26658 raft_consensus.cc:1261] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:55.246943 29213 raft_consensus.cc:1261] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:55.247310 319 consensus_queue.cc:1035] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:55.247490 32508 consensus_queue.cc:1035] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:55.249248 32508 raft_consensus.cc:2932] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:55.249603 29213 raft_consensus.cc:2932] T c455f182f874431a88a46a8aff2b1675 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:55.249609 26658 raft_consensus.cc:2932] T c455f182f874431a88a46a8aff2b1675 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:55.252213 26509 catalog_manager.cc:5484] T c455f182f874431a88a46a8aff2b1675 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:55.254973 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet c455f182f874431a88a46a8aff2b1675 with delete_type TABLET_DATA_TOMBSTONED (TS 7c693d0933c64d9babc07f404df45cd1 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:55.255698 787 tablet_replica.cc:330] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:55.255856 787 raft_consensus.cc:2229] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.256049 787 raft_consensus.cc:2258] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.256505 787 ts_tablet_manager.cc:1875] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:55.258517 787 ts_tablet_manager.cc:1888] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:55.258646 787 log.cc:1196] T c455f182f874431a88a46a8aff2b1675 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/c455f182f874431a88a46a8aff2b1675
I20240502 01:58:55.259218 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet c455f182f874431a88a46a8aff2b1675 (table rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]) successfully deleted
I20240502 01:58:55.325814 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:55.330335 29489 raft_consensus.cc:1261] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:55.330744 32364 consensus_queue.cc:1035] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.331643 26927 raft_consensus.cc:1261] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:55.332088 32364 consensus_queue.cc:1035] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.333199 832 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:55.333339 26513 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]). Expected version 1 got 0
I20240502 01:58:55.333550 832 tablet.cc:1620] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:55.335605 26513 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]). Expected version 1 got 0
I20240502 01:58:55.336138 26506 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]). Expected version 1 got 0
I20240502 01:58:55.337321 834 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:55.337599 834 tablet.cc:1620] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:55.341089 26506 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61] alter complete (version 1)
I20240502 01:58:55.343075 836 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:55.343358 26506 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61]). Expected version 1 got 0
I20240502 01:58:55.343372 836 tablet.cc:1620] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:55.357096 27044 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:55.357982 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.359434 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.359862 29468 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.360807 29330 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:55.365897 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:55.371340 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:55.412372 29059 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:55.421754 744 consensus_queue.cc:1035] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.424556 744 consensus_queue.cc:1035] T bb13643f981847c4bb480b74a4764c4c P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.438997 744 consensus_queue.cc:1035] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.442075 339 consensus_queue.cc:1035] T c3d890af39194df197564cb91e80297e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.447575 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:55.452665 832 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:55.452953 832 tablet.cc:1620] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:55.453703 834 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:55.453979 834 tablet.cc:1620] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:55.456130 27063 consensus_queue.cc:237] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.456694 836 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:55.456960 836 tablet.cc:1620] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:55.457305 29489 raft_consensus.cc:1261] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:55.457639 32192 consensus_queue.cc:1035] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:55.457809 26927 raft_consensus.cc:1261] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:55.458256 32192 consensus_queue.cc:1035] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:55.459558 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=374f1d99d81d406495f36a173dee5e61] alter complete (version 2)
I20240502 01:58:55.460312 32573 raft_consensus.cc:2932] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.461375 29489 raft_consensus.cc:2932] T dfdd6d487b57479cb48a92bc770947ce P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.461400 26927 raft_consensus.cc:2932] T dfdd6d487b57479cb48a92bc770947ce P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.463836 26509 catalog_manager.cc:5484] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.466094 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet dfdd6d487b57479cb48a92bc770947ce with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:55.466435 873 tablet_replica.cc:330] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:55.466598 873 raft_consensus.cc:2229] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.466745 873 raft_consensus.cc:2258] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.467612 873 ts_tablet_manager.cc:1875] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:55.469257 873 ts_tablet_manager.cc:1888] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:55.469322 873 log.cc:1196] T dfdd6d487b57479cb48a92bc770947ce P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/dfdd6d487b57479cb48a92bc770947ce
I20240502 01:58:55.469816 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet dfdd6d487b57479cb48a92bc770947ce (table rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]) successfully deleted
I20240502 01:58:55.545511 29330 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 01:58:55.545679 27044 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:55.548067 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.560551 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:55.562189 29468 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.562712 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.567664 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:55.570173 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:55.576709 832 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:55.578126 834 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:55.583663 836 tablet.cc:1722] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:55.582870 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=374f1d99d81d406495f36a173dee5e61] alter complete (version 3)
I20240502 01:58:55.604261 29059 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:55.634490 339 consensus_queue.cc:1035] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.639405 339 consensus_queue.cc:1035] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:55.651016 26793 consensus_queue.cc:237] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.652112 29213 raft_consensus.cc:1261] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:55.652163 26658 raft_consensus.cc:1261] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:55.652489 747 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:55.652675 32646 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:55.654456 948 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:55.654804 29213 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:55.654812 26658 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:55.657085 26512 catalog_manager.cc:5484] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:55.660316 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet dc99f8bfb06c4bcda6111b218125826c with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:55.661579 949 tablet_replica.cc:330] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:55.661896 949 raft_consensus.cc:2229] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.662189 949 raft_consensus.cc:2258] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.662127 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:55.662915 949 ts_tablet_manager.cc:1875] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:55.664994 949 ts_tablet_manager.cc:1888] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:55.665068 949 log.cc:1196] T dc99f8bfb06c4bcda6111b218125826c P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/dc99f8bfb06c4bcda6111b218125826c
I20240502 01:58:55.665899 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet dc99f8bfb06c4bcda6111b218125826c (table rebalance_test_table_19 [id=cdfbad05a8f14dc6bceb10ffc9f00c82]) successfully deleted
I20240502 01:58:55.667685 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 30eb01a4663b473faa96285b3eb7871a (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.667701 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 30eb01a4663b473faa96285b3eb7871a (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.667946 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30eb01a4663b473faa96285b3eb7871a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.667973 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30eb01a4663b473faa96285b3eb7871a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.668066 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 7f701a3437f84a84b6e3290266d24c94 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.668252 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f701a3437f84a84b6e3290266d24c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.668256 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 30eb01a4663b473faa96285b3eb7871a (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.668483 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30eb01a4663b473faa96285b3eb7871a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.670570 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 7f701a3437f84a84b6e3290266d24c94 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.670662 950 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:55.670828 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f701a3437f84a84b6e3290266d24c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.670948 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 04d8d8cfaea04803b38a8c8a2b2e5c39 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.671187 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d8d8cfaea04803b38a8c8a2b2e5c39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.671773 950 tablet_bootstrap.cc:654] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.672609 954 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:55.672768 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 04d8d8cfaea04803b38a8c8a2b2e5c39 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.673000 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d8d8cfaea04803b38a8c8a2b2e5c39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.673173 955 tablet_bootstrap.cc:492] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:55.673488 954 tablet_bootstrap.cc:654] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.673635 952 tablet_bootstrap.cc:492] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:55.673444 951 tablet_bootstrap.cc:492] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:55.674103 955 tablet_bootstrap.cc:654] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.674345 950 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:55.674422 950 ts_tablet_manager.cc:1367] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:55.674830 950 raft_consensus.cc:348] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.674932 950 raft_consensus.cc:374] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.674962 950 raft_consensus.cc:729] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.675083 950 consensus_queue.cc:260] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.675166 955 tablet_bootstrap.cc:492] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:55.675235 955 ts_tablet_manager.cc:1367] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:55.675338 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 7f701a3437f84a84b6e3290266d24c94 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.675566 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f701a3437f84a84b6e3290266d24c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.675631 955 raft_consensus.cc:348] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.675714 955 raft_consensus.cc:374] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.675741 955 raft_consensus.cc:729] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.675849 955 consensus_queue.cc:260] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.675897 950 ts_tablet_manager.cc:1398] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:55.676051 950 tablet_bootstrap.cc:492] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:55.676100 955 ts_tablet_manager.cc:1398] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:55.676863 950 tablet_bootstrap.cc:654] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.676898 958 tablet_bootstrap.cc:492] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:55.677224 952 tablet_bootstrap.cc:654] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.677979 958 tablet_bootstrap.cc:654] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.678980 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 04d8d8cfaea04803b38a8c8a2b2e5c39 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:55.679075 951 tablet_bootstrap.cc:654] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.679208 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d8d8cfaea04803b38a8c8a2b2e5c39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.680212 954 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:55.680300 954 ts_tablet_manager.cc:1367] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:58:55.680512 952 tablet_bootstrap.cc:492] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:55.680588 952 ts_tablet_manager.cc:1367] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 01:58:55.680670 958 tablet_bootstrap.cc:492] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:55.680748 958 ts_tablet_manager.cc:1367] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:55.680712 954 raft_consensus.cc:348] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.680799 954 raft_consensus.cc:374] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.680826 954 raft_consensus.cc:729] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.680922 954 consensus_queue.cc:260] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.681201 954 ts_tablet_manager.cc:1398] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:55.681169 958 raft_consensus.cc:348] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.681258 958 raft_consensus.cc:374] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.681296 958 raft_consensus.cc:729] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.681401 959 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:55.681406 958 consensus_queue.cc:260] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.681600 950 tablet_bootstrap.cc:492] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:55.681664 950 ts_tablet_manager.cc:1367] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 01:58:55.681666 958 ts_tablet_manager.cc:1398] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:55.682080 950 raft_consensus.cc:348] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.682173 950 raft_consensus.cc:374] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.682199 950 raft_consensus.cc:729] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.682309 950 consensus_queue.cc:260] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.682716 959 tablet_bootstrap.cc:654] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.683681 959 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:55.683760 959 ts_tablet_manager.cc:1367] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:55.683962 952 raft_consensus.cc:348] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.684057 952 raft_consensus.cc:374] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.684087 952 raft_consensus.cc:729] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.684170 959 raft_consensus.cc:348] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.684252 959 raft_consensus.cc:374] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.684226 952 consensus_queue.cc:260] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.684280 959 raft_consensus.cc:729] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.684381 959 consensus_queue.cc:260] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.684494 952 ts_tablet_manager.cc:1398] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:58:55.684628 959 ts_tablet_manager.cc:1398] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:55.684880 950 ts_tablet_manager.cc:1398] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:55.685109 954 tablet_bootstrap.cc:492] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:55.687533 951 tablet_bootstrap.cc:492] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:55.687789 951 ts_tablet_manager.cc:1367] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:55.687830 26510 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:55.687999 26510 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:55.688385 951 raft_consensus.cc:348] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.688546 951 raft_consensus.cc:374] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.688712 951 raft_consensus.cc:729] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.688931 954 tablet_bootstrap.cc:654] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.688920 951 consensus_queue.cc:260] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.689823 951 ts_tablet_manager.cc:1398] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:55.690958 954 tablet_bootstrap.cc:492] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:55.691051 954 ts_tablet_manager.cc:1367] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:58:55.691452 954 raft_consensus.cc:348] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.691552 954 raft_consensus.cc:374] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.691583 954 raft_consensus.cc:729] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.691717 954 consensus_queue.cc:260] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.691965 954 ts_tablet_manager.cc:1398] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:55.701202 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet f2f5b10edbb943509186ea012abdcaf4
I20240502 01:58:55.701566 765 raft_consensus.cc:482] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.701649 765 raft_consensus.cc:504] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.701987 765 leader_election.cc:288] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:55.702105 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f5b10edbb943509186ea012abdcaf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:55 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:55.702466 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f5b10edbb943509186ea012abdcaf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:55 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:55.702666 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f5b10edbb943509186ea012abdcaf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:55 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:55.702713 787 tablet_replica.cc:330] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:55.702795 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f701a3437f84a84b6e3290266d24c94" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:55.702827 787 raft_consensus.cc:2229] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.702889 29079 raft_consensus.cc:2454] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:55.702927 787 raft_consensus.cc:2258] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.703164 967 tablet_replica.cc:330] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:55.703291 967 raft_consensus.cc:2229] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:55.703398 967 raft_consensus.cc:2258] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.703469 787 ts_tablet_manager.cc:1875] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:55.703600 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f701a3437f84a84b6e3290266d24c94" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:58:55.703698 26658 raft_consensus.cc:2454] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:55.704169 29285 leader_election.cc:302] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:55.704363 765 raft_consensus.cc:2781] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.704409 765 raft_consensus.cc:482] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.704442 765 raft_consensus.cc:3037] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.705108 966 tablet_replica.cc:330] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:55.705247 966 raft_consensus.cc:2229] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:55.705333 765 raft_consensus.cc:504] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.705478 966 raft_consensus.cc:2258] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:55.705670 765 leader_election.cc:288] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:55.705708 787 ts_tablet_manager.cc:1888] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:55.705763 787 log.cc:1196] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/f2f5b10edbb943509186ea012abdcaf4
I20240502 01:58:55.706012 787 ts_tablet_manager.cc:1909] T f2f5b10edbb943509186ea012abdcaf4 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:55.706357 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f701a3437f84a84b6e3290266d24c94" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:55.706436 966 ts_tablet_manager.cc:1875] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:55.706452 26658 raft_consensus.cc:3037] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.706430 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f701a3437f84a84b6e3290266d24c94" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:55.706521 29079 raft_consensus.cc:3037] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.707051 26658 raft_consensus.cc:2454] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:55.707392 29079 raft_consensus.cc:2454] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:55.707579 29285 leader_election.cc:302] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:55.707917 765 raft_consensus.cc:2781] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.707998 765 raft_consensus.cc:686] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:55.708102 765 consensus_queue.cc:237] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:55.708396 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table_new_name [id=374f1d99d81d406495f36a173dee5e61]) successfully deleted
I20240502 01:58:55.704272 967 ts_tablet_manager.cc:1875] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:55.709259 966 ts_tablet_manager.cc:1888] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:55.709319 966 log.cc:1196] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/f2f5b10edbb943509186ea012abdcaf4
I20240502 01:58:55.709550 966 ts_tablet_manager.cc:1909] T f2f5b10edbb943509186ea012abdcaf4 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:55.709908 26510 catalog_manager.cc:5484] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:55.710239 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table_new_name [id=374f1d99d81d406495f36a173dee5e61]) successfully deleted
I20240502 01:58:55.711092 967 ts_tablet_manager.cc:1888] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:55.711213 967 log.cc:1196] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/f2f5b10edbb943509186ea012abdcaf4
I20240502 01:58:55.711735 967 ts_tablet_manager.cc:1909] T f2f5b10edbb943509186ea012abdcaf4 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:55.712355 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet f2f5b10edbb943509186ea012abdcaf4 (table rebalancer_dynamic_table_new_name [id=374f1d99d81d406495f36a173dee5e61]) successfully deleted
I20240502 01:58:55.760229 529 raft_consensus.cc:482] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.760334 529 raft_consensus.cc:504] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.760680 529 leader_election.cc:288] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:55.761000 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d8d8cfaea04803b38a8c8a2b2e5c39" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:55.761039 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d8d8cfaea04803b38a8c8a2b2e5c39" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:55.761129 29213 raft_consensus.cc:2454] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:55.761166 27063 raft_consensus.cc:2454] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:55.761411 29015 leader_election.cc:302] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:55.761627 529 raft_consensus.cc:2781] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.761672 529 raft_consensus.cc:482] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.761706 529 raft_consensus.cc:3037] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.762498 529 raft_consensus.cc:504] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.762792 529 leader_election.cc:288] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:55.763090 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d8d8cfaea04803b38a8c8a2b2e5c39" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:55.763106 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d8d8cfaea04803b38a8c8a2b2e5c39" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:55.763182 27063 raft_consensus.cc:3037] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.763218 29213 raft_consensus.cc:3037] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.763880 27063 raft_consensus.cc:2454] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:55.764087 29213 raft_consensus.cc:2454] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:55.764268 29016 leader_election.cc:302] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:55.764891 969 raft_consensus.cc:2781] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.764978 969 raft_consensus.cc:686] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:55.765097 969 consensus_queue.cc:237] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:55.766866 26510 catalog_manager.cc:5484] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:55.803226 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:55.808467 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 5d08c84bae3e4f33b468cfacc181633b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5]), partition=
I20240502 01:58:55.808526 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 5d08c84bae3e4f33b468cfacc181633b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5]), partition=
I20240502 01:58:55.808730 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d08c84bae3e4f33b468cfacc181633b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.808732 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d08c84bae3e4f33b468cfacc181633b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.810806 954 tablet_bootstrap.cc:492] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:55.810858 950 tablet_bootstrap.cc:492] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:55.811640 954 tablet_bootstrap.cc:654] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.811764 950 tablet_bootstrap.cc:654] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.812633 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 5d08c84bae3e4f33b468cfacc181633b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5]), partition=
I20240502 01:58:55.812913 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d08c84bae3e4f33b468cfacc181633b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:55.813531 950 tablet_bootstrap.cc:492] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:55.813607 950 ts_tablet_manager.cc:1367] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:55.813843 954 tablet_bootstrap.cc:492] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:55.814181 954 ts_tablet_manager.cc:1367] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:55.814249 950 raft_consensus.cc:348] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.814342 950 raft_consensus.cc:374] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.814369 950 raft_consensus.cc:729] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.814472 950 consensus_queue.cc:260] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.814733 950 ts_tablet_manager.cc:1398] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:55.815068 951 tablet_bootstrap.cc:492] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:55.816103 951 tablet_bootstrap.cc:654] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:55.814913 954 raft_consensus.cc:348] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.816637 954 raft_consensus.cc:374] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.816712 954 raft_consensus.cc:729] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.817093 951 tablet_bootstrap.cc:492] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:55.817180 951 ts_tablet_manager.cc:1367] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:55.817579 951 raft_consensus.cc:348] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.817724 951 raft_consensus.cc:374] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:55.817767 951 raft_consensus.cc:729] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:55.817902 951 consensus_queue.cc:260] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.818190 951 ts_tablet_manager.cc:1398] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:55.818481 954 consensus_queue.cc:260] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.818838 954 ts_tablet_manager.cc:1398] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:55.888706 32364 raft_consensus.cc:482] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.888818 32364 raft_consensus.cc:504] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.889201 32364 leader_election.cc:288] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:55.889493 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb01a4663b473faa96285b3eb7871a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:55.889513 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb01a4663b473faa96285b3eb7871a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:55.889626 29489 raft_consensus.cc:2454] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:55.889633 26793 raft_consensus.cc:2454] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:55.889894 29286 leader_election.cc:302] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.890074 32364 raft_consensus.cc:2781] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.890132 32364 raft_consensus.cc:482] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.890169 32364 raft_consensus.cc:3037] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.891093 32364 raft_consensus.cc:504] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.891414 32364 leader_election.cc:288] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:55.891695 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb01a4663b473faa96285b3eb7871a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:55.891745 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb01a4663b473faa96285b3eb7871a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:55.891799 29489 raft_consensus.cc:3037] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.891850 26793 raft_consensus.cc:3037] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.892791 26793 raft_consensus.cc:2454] T 30eb01a4663b473faa96285b3eb7871a P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:55.893108 29489 raft_consensus.cc:2454] T 30eb01a4663b473faa96285b3eb7871a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:55.893154 29285 leader_election.cc:302] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:55.893404 32364 raft_consensus.cc:2781] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.893486 32364 raft_consensus.cc:686] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:55.893621 32364 consensus_queue.cc:237] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:55.895423 26508 catalog_manager.cc:5484] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:55.916260 339 raft_consensus.cc:482] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:55.916352 339 raft_consensus.cc:504] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.916698 339 leader_election.cc:288] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:55.917344 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d08c84bae3e4f33b468cfacc181633b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:55.917481 29350 raft_consensus.cc:2454] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.917733 29419 leader_election.cc:302] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:55.917795 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d08c84bae3e4f33b468cfacc181633b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:55.917915 29079 raft_consensus.cc:2454] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:55.918102 339 raft_consensus.cc:2781] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:55.918160 339 raft_consensus.cc:482] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:55.918197 339 raft_consensus.cc:3037] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.919201 339 raft_consensus.cc:504] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.919559 339 leader_election.cc:288] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:55.920349 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d08c84bae3e4f33b468cfacc181633b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:55.920446 29350 raft_consensus.cc:3037] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.921386 29350 raft_consensus.cc:2454] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.921926 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d08c84bae3e4f33b468cfacc181633b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:55.922061 29079 raft_consensus.cc:3037] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:55.923008 29079 raft_consensus.cc:2454] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:55.923393 29419 leader_election.cc:302] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:55.923900 339 raft_consensus.cc:2781] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:55.923995 339 raft_consensus.cc:686] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:55.924115 339 consensus_queue.cc:237] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:55.926540 26506 catalog_manager.cc:5484] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:55.952589 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.955348 29330 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.959065 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:55.959550 29468 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:55.961935 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:55.969895 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:55.994175 27044 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:56.009213 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:56.032200 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:56.036401 29079 raft_consensus.cc:1261] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.036401 29350 raft_consensus.cc:1261] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.036772 339 consensus_queue.cc:1035] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.037053 339 consensus_queue.cc:1035] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.039714 26506 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 5d08c84bae3e4f33b468cfacc181633b (table rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5]). Expected version 1 got 0
I20240502 01:58:56.040498 834 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:56.040822 834 tablet.cc:1620] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:56.041502 1036 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:56.041697 26506 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 5d08c84bae3e4f33b468cfacc181633b (table rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5]). Expected version 1 got 0
I20240502 01:58:56.041791 1036 tablet.cc:1620] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:56.043051 832 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:56.043323 832 tablet.cc:1620] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:56.046483 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5] alter complete (version 1)
I20240502 01:58:56.057457 26930 consensus_queue.cc:237] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:56.058568 29079 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:56.058601 26658 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:56.058959 1048 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:56.059157 319 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:56.060747 319 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.061385 26658 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.062332 29079 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.063761 26509 catalog_manager.cc:5484] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.066251 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 5515469fdf494900ae20f20fdfca587c with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:58:56.066445 949 tablet_replica.cc:330] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:58:56.066574 949 raft_consensus.cc:2229] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:56.066707 949 raft_consensus.cc:2258] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:56.067317 949 ts_tablet_manager.cc:1875] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:56.069145 949 ts_tablet_manager.cc:1888] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:56.069195 949 log.cc:1196] T 5515469fdf494900ae20f20fdfca587c P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/5515469fdf494900ae20f20fdfca587c
I20240502 01:58:56.069595 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 5515469fdf494900ae20f20fdfca587c (table rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]) successfully deleted
I20240502 01:58:56.149278 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:56.158411 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.167290 834 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:56.167609 834 tablet.cc:1620] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:58:56.167886 832 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:56.168159 832 tablet.cc:1620] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:56.169361 1036 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:56.169629 1036 tablet.cc:1620] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:56.171253 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:56.175140 29330 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.177767 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=8d833c89ca324358aca394cc8648eae5] alter complete (version 2)
I20240502 01:58:56.178378 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.179092 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:56.179340 29468 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:56.181259 27044 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:56.185953 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:56.200402 969 consensus_queue.cc:1035] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.205122 969 consensus_queue.cc:1035] T 04d8d8cfaea04803b38a8c8a2b2e5c39 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.205265 32364 consensus_queue.cc:1035] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.207891 32364 consensus_queue.cc:1035] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.256984 27064 consensus_queue.cc:237] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:56.258121 29079 raft_consensus.cc:1261] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:56.258121 26661 raft_consensus.cc:1261] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:56.258460 32573 consensus_queue.cc:1035] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:56.258638 32192 consensus_queue.cc:1035] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:56.260511 32573 raft_consensus.cc:2932] T e24e0bd4a1df4fed876e1c4c50af7d70 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.261155 29079 raft_consensus.cc:2932] T e24e0bd4a1df4fed876e1c4c50af7d70 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.261265 26661 raft_consensus.cc:2932] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.263727 26506 catalog_manager.cc:5484] T e24e0bd4a1df4fed876e1c4c50af7d70 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:56.266482 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet e24e0bd4a1df4fed876e1c4c50af7d70 with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:56.266896 1133 tablet_replica.cc:330] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:56.267284 1133 raft_consensus.cc:2229] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:56.267647 1133 raft_consensus.cc:2258] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:56.268205 1133 ts_tablet_manager.cc:1875] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:56.270783 1133 ts_tablet_manager.cc:1888] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:56.270860 1133 log.cc:1196] T e24e0bd4a1df4fed876e1c4c50af7d70 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/e24e0bd4a1df4fed876e1c4c50af7d70
I20240502 01:58:56.271338 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet e24e0bd4a1df4fed876e1c4c50af7d70 (table rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]) successfully deleted
I20240502 01:58:56.281811 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:56.288388 834 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:56.288707 1036 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:56.289261 832 tablet.cc:1722] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:56.293314 26514 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=8d833c89ca324358aca394cc8648eae5] alter complete (version 3)
I20240502 01:58:56.353549 29330 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.359344 29468 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:56.359724 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:56.362946 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.369911 27044 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:56.378108 26641 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.383991 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:56.396136 26513 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:56.396266 26513 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:56.398357 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 5d08c84bae3e4f33b468cfacc181633b
I20240502 01:58:56.398962 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:56.399976 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d08c84bae3e4f33b468cfacc181633b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:56 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:56.400221 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d08c84bae3e4f33b468cfacc181633b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:56 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:58:56.400442 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d08c84bae3e4f33b468cfacc181633b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:56 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:56.400720 1185 tablet_replica.cc:330] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:58:56.400909 1185 raft_consensus.cc:2229] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:56.401134 1185 raft_consensus.cc:2258] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:56.401713 1185 ts_tablet_manager.cc:1875] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:56.404155 1185 ts_tablet_manager.cc:1888] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:56.404314 1185 log.cc:1196] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/5d08c84bae3e4f33b468cfacc181633b
I20240502 01:58:56.404738 1185 ts_tablet_manager.cc:1909] T 5d08c84bae3e4f33b468cfacc181633b P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:58:56.405480 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 5d08c84bae3e4f33b468cfacc181633b (table rebalancer_dynamic_table_new_name [id=8d833c89ca324358aca394cc8648eae5]) successfully deleted
I20240502 01:58:56.418579 1184 tablet_replica.cc:330] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:56.421830 1184 raft_consensus.cc:2229] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:56.421954 1184 raft_consensus.cc:2258] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:56.422986 1184 ts_tablet_manager.cc:1875] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:56.423360 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:56.424963 1184 ts_tablet_manager.cc:1888] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:56.425011 1184 log.cc:1196] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/5d08c84bae3e4f33b468cfacc181633b
I20240502 01:58:56.425235 1184 ts_tablet_manager.cc:1909] T 5d08c84bae3e4f33b468cfacc181633b P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:56.429008 1186 tablet_replica.cc:330] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:56.429141 1186 raft_consensus.cc:2229] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:56.429251 1186 raft_consensus.cc:2258] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:56.429513 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 5d08c84bae3e4f33b468cfacc181633b (table rebalancer_dynamic_table_new_name [id=8d833c89ca324358aca394cc8648eae5]) successfully deleted
I20240502 01:58:56.429590 26775 tablet_service.cc:1467] Processing CreateTablet for tablet d216b8ccd8ea43df85c1013cdaf36564 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.429859 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d216b8ccd8ea43df85c1013cdaf36564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.430132 29059 tablet_service.cc:1467] Processing CreateTablet for tablet d216b8ccd8ea43df85c1013cdaf36564 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.430388 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d216b8ccd8ea43df85c1013cdaf36564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.430441 29330 tablet_service.cc:1467] Processing CreateTablet for tablet d2ef5e17842d47ccac3b908e9b389384 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.430653 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ef5e17842d47ccac3b908e9b389384. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.431135 765 consensus_queue.cc:1035] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.432832 29195 tablet_service.cc:1467] Processing CreateTablet for tablet d2ef5e17842d47ccac3b908e9b389384 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.433072 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ef5e17842d47ccac3b908e9b389384. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.433758 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 7ca17309584740f28f6a9163a2c2198c (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.433769 26641 tablet_service.cc:1467] Processing CreateTablet for tablet d216b8ccd8ea43df85c1013cdaf36564 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.433995 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ca17309584740f28f6a9163a2c2198c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.434068 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d216b8ccd8ea43df85c1013cdaf36564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.434595 26640 tablet_service.cc:1467] Processing CreateTablet for tablet d2ef5e17842d47ccac3b908e9b389384 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.434605 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 7ca17309584740f28f6a9163a2c2198c (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.434820 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2ef5e17842d47ccac3b908e9b389384. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.434860 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ca17309584740f28f6a9163a2c2198c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.436859 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 7ca17309584740f28f6a9163a2c2198c (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:56.437043 1193 tablet_bootstrap.cc:492] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:56.437094 1189 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:56.437119 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ca17309584740f28f6a9163a2c2198c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.437594 1192 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:56.437949 1193 tablet_bootstrap.cc:654] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.438719 1190 tablet_bootstrap.cc:492] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:56.439393 1189 tablet_bootstrap.cc:654] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.440326 1187 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:56.438500 1192 tablet_bootstrap.cc:654] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.442066 1190 tablet_bootstrap.cc:654] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.442731 1186 ts_tablet_manager.cc:1875] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:56.442752 1187 tablet_bootstrap.cc:654] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.444154 1190 tablet_bootstrap.cc:492] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:56.444276 1190 ts_tablet_manager.cc:1367] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 01:58:56.444741 1186 ts_tablet_manager.cc:1888] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:56.444810 1186 log.cc:1196] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/5d08c84bae3e4f33b468cfacc181633b
I20240502 01:58:56.445057 1186 ts_tablet_manager.cc:1909] T 5d08c84bae3e4f33b468cfacc181633b P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:56.445035 1190 raft_consensus.cc:348] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.445308 1187 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:56.445402 1195 tablet_bootstrap.cc:492] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:56.445443 1187 ts_tablet_manager.cc:1367] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:56.445873 1187 raft_consensus.cc:348] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.446048 1190 raft_consensus.cc:374] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.446134 1190 raft_consensus.cc:729] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.446322 1190 consensus_queue.cc:260] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.446377 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 5d08c84bae3e4f33b468cfacc181633b (table rebalancer_dynamic_table_new_name [id=8d833c89ca324358aca394cc8648eae5]) successfully deleted
I20240502 01:58:56.446713 1190 ts_tablet_manager.cc:1398] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:56.446818 1187 raft_consensus.cc:374] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.446906 1187 raft_consensus.cc:729] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.447085 1187 consensus_queue.cc:260] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.448179 1187 ts_tablet_manager.cc:1398] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:56.449193 32364 consensus_queue.cc:1035] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.449801 1189 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:56.449889 1189 ts_tablet_manager.cc:1367] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:58:56.451493 1195 tablet_bootstrap.cc:654] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.451654 1189 raft_consensus.cc:348] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.452287 1189 raft_consensus.cc:374] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.452376 1189 raft_consensus.cc:729] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.453087 1189 consensus_queue.cc:260] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.453433 1189 ts_tablet_manager.cc:1398] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:56.454200 1197 tablet_bootstrap.cc:492] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:56.455152 1197 tablet_bootstrap.cc:654] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.455693 1192 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:56.455842 1192 ts_tablet_manager.cc:1367] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 01:58:56.455960 1191 tablet_bootstrap.cc:492] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:56.456310 1195 tablet_bootstrap.cc:492] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:56.456316 1192 raft_consensus.cc:348] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.456434 1192 raft_consensus.cc:374] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.456475 1192 raft_consensus.cc:729] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.456563 1195 ts_tablet_manager.cc:1367] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:58:56.456595 1192 consensus_queue.cc:260] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.456869 1192 ts_tablet_manager.cc:1398] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:56.457084 1191 tablet_bootstrap.cc:654] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.457345 1195 raft_consensus.cc:348] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.457435 1195 raft_consensus.cc:374] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.457464 1195 raft_consensus.cc:729] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.457595 1195 consensus_queue.cc:260] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.457883 1195 ts_tablet_manager.cc:1398] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:56.458308 1191 tablet_bootstrap.cc:492] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:56.458477 1191 ts_tablet_manager.cc:1367] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:56.458588 1192 tablet_bootstrap.cc:492] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:56.458633 1193 tablet_bootstrap.cc:492] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:56.458734 1193 ts_tablet_manager.cc:1367] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240502 01:58:56.459177 1193 raft_consensus.cc:348] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.459283 1193 raft_consensus.cc:374] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.459322 1193 raft_consensus.cc:729] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.459468 1192 tablet_bootstrap.cc:654] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.459439 1193 consensus_queue.cc:260] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.459728 1193 ts_tablet_manager.cc:1398] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:56.459695 1191 raft_consensus.cc:348] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.459794 1191 raft_consensus.cc:374] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.459825 1191 raft_consensus.cc:729] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.459950 1191 consensus_queue.cc:260] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.461351 1192 tablet_bootstrap.cc:492] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:56.461437 1192 ts_tablet_manager.cc:1367] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:58:56.461912 1191 ts_tablet_manager.cc:1398] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:58:56.462671 1192 raft_consensus.cc:348] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.462765 1192 raft_consensus.cc:374] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.462795 1192 raft_consensus.cc:729] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.462903 1192 consensus_queue.cc:260] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.463151 1192 ts_tablet_manager.cc:1398] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 01:58:56.463913 1197 tablet_bootstrap.cc:492] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:56.464051 1197 ts_tablet_manager.cc:1367] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:58:56.465075 1197 raft_consensus.cc:348] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.465188 1197 raft_consensus.cc:374] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.465219 1197 raft_consensus.cc:729] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.465322 1197 consensus_queue.cc:260] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.466584 1197 ts_tablet_manager.cc:1398] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:56.469307 31766 raft_consensus.cc:482] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:56.469396 31766 raft_consensus.cc:504] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.469842 31766 leader_election.cc:288] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:56.470356 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ef5e17842d47ccac3b908e9b389384" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:56.470476 29350 raft_consensus.cc:2454] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:56.470846 26592 leader_election.cc:302] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:56.471407 31766 raft_consensus.cc:2781] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:56.471462 31766 raft_consensus.cc:482] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:56.471498 31766 raft_consensus.cc:3037] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.472321 31766 raft_consensus.cc:504] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.472584 31766 leader_election.cc:288] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:56.472865 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ef5e17842d47ccac3b908e9b389384" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:56.472970 29350 raft_consensus.cc:3037] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.473917 29350 raft_consensus.cc:2454] T d2ef5e17842d47ccac3b908e9b389384 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:56.474205 26592 leader_election.cc:302] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:56.474370 1217 raft_consensus.cc:2781] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:56.474505 1217 raft_consensus.cc:686] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:56.474635 1217 consensus_queue.cc:237] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.476445 26510 catalog_manager.cc:5484] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:56.478332 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ef5e17842d47ccac3b908e9b389384" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:56.478456 29213 raft_consensus.cc:2454] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:56.478483 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2ef5e17842d47ccac3b908e9b389384" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:56.478586 29216 raft_consensus.cc:3037] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.479488 29216 raft_consensus.cc:2454] T d2ef5e17842d47ccac3b908e9b389384 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:56.486588 29079 consensus_queue.cc:237] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:56.487706 27064 raft_consensus.cc:1261] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:56.487854 26793 raft_consensus.cc:1261] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:56.488221 1105 consensus_queue.cc:1035] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:56.488412 969 consensus_queue.cc:1035] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:56.490069 529 raft_consensus.cc:2932] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.490491 26793 raft_consensus.cc:2932] T 8e66776cedd8407dad9c187b0b04f073 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.491736 27064 raft_consensus.cc:2932] T 8e66776cedd8407dad9c187b0b04f073 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.492506 26510 catalog_manager.cc:5484] T 8e66776cedd8407dad9c187b0b04f073 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.496173 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e66776cedd8407dad9c187b0b04f073 with delete_type TABLET_DATA_TOMBSTONED (TS 0d83fdd1e6e04cf5a5602994d1dbc0df not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:56.496742 1232 tablet_replica.cc:330] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:56.496881 1232 raft_consensus.cc:2229] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:56.497056 1232 raft_consensus.cc:2258] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:56.497440 1232 ts_tablet_manager.cc:1875] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:56.499223 1232 ts_tablet_manager.cc:1888] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:56.499282 1232 log.cc:1196] T 8e66776cedd8407dad9c187b0b04f073 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/8e66776cedd8407dad9c187b0b04f073
I20240502 01:58:56.499752 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 8e66776cedd8407dad9c187b0b04f073 (table rebalancer_extra_table_2 [id=1cf4692eb92a46f08f5033d8b17962f2]) successfully deleted
I20240502 01:58:56.501122 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:56.506183 29059 tablet_service.cc:1467] Processing CreateTablet for tablet e8731e59b86f44ecaa2a0d5a955244cb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=81041992874547a38070159676060e8b]), partition=
I20240502 01:58:56.506256 26908 tablet_service.cc:1467] Processing CreateTablet for tablet e8731e59b86f44ecaa2a0d5a955244cb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=81041992874547a38070159676060e8b]), partition=
I20240502 01:58:56.506512 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8731e59b86f44ecaa2a0d5a955244cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.506875 29330 tablet_service.cc:1467] Processing CreateTablet for tablet e8731e59b86f44ecaa2a0d5a955244cb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=81041992874547a38070159676060e8b]), partition=
I20240502 01:58:56.507058 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8731e59b86f44ecaa2a0d5a955244cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.508301 1195 tablet_bootstrap.cc:492] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:56.508723 1190 tablet_bootstrap.cc:492] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:56.506412 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8731e59b86f44ecaa2a0d5a955244cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:56.509258 1195 tablet_bootstrap.cc:654] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.509483 1190 tablet_bootstrap.cc:654] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.510449 1195 tablet_bootstrap.cc:492] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:56.510524 1195 ts_tablet_manager.cc:1367] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:56.510952 1195 raft_consensus.cc:348] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.511042 1195 raft_consensus.cc:374] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.511070 1195 raft_consensus.cc:729] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.511183 1195 consensus_queue.cc:260] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.511219 1189 tablet_bootstrap.cc:492] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:56.511351 1190 tablet_bootstrap.cc:492] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:56.511425 1190 ts_tablet_manager.cc:1367] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:56.511438 1195 ts_tablet_manager.cc:1398] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:56.511854 1190 raft_consensus.cc:348] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.511945 1190 raft_consensus.cc:374] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.511981 1190 raft_consensus.cc:729] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.512027 1189 tablet_bootstrap.cc:654] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:56.512099 1190 consensus_queue.cc:260] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.512365 1190 ts_tablet_manager.cc:1398] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:58:56.513110 1189 tablet_bootstrap.cc:492] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:56.513185 1189 ts_tablet_manager.cc:1367] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:56.513557 1189 raft_consensus.cc:348] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.513643 1189 raft_consensus.cc:374] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:56.513670 1189 raft_consensus.cc:729] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:56.513788 1189 consensus_queue.cc:260] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.514070 1189 ts_tablet_manager.cc:1398] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:56.573135 1217 raft_consensus.cc:482] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:56.573230 1217 raft_consensus.cc:504] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.573606 1217 leader_election.cc:288] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:56.573922 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:56.573963 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:56.574069 26793 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:56.574112 29079 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:56.574415 26592 leader_election.cc:302] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:56.574669 1217 raft_consensus.cc:2781] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:56.574743 1217 raft_consensus.cc:482] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:56.574795 1217 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.575935 1217 raft_consensus.cc:504] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.576277 1217 leader_election.cc:288] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:56.576681 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:56.576797 29079 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.577757 29079 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:56.578063 26592 leader_election.cc:302] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:56.578234 1217 raft_consensus.cc:2781] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:56.578336 1217 raft_consensus.cc:686] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:56.578456 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:56.578462 1217 consensus_queue.cc:237] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:58:56.578545 26793 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.579478 26793 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:56.581174 26510 catalog_manager.cc:5484] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:58:56.584620 27064 consensus_queue.cc:237] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:56.586153 26930 raft_consensus.cc:1261] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.586864 32192 consensus_queue.cc:1035] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:56.587074 26995 consensus_peers.cc:485] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47f84b6ffb074bc682aa7b087b8c3614. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.587369 26661 raft_consensus.cc:1261] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.587716 32192 consensus_queue.cc:1035] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.589414 32192 raft_consensus.cc:2932] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.589825 26661 raft_consensus.cc:2932] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.591444 26930 raft_consensus.cc:2932] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.591768 26510 catalog_manager.cc:5484] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.594113 32364 raft_consensus.cc:482] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:56.594213 32364 raft_consensus.cc:504] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.594569 32364 leader_election.cc:288] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:56.594867 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8731e59b86f44ecaa2a0d5a955244cb" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:56.594970 29079 raft_consensus.cc:2454] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:56.595389 29283 leader_election.cc:302] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:56.595572 32364 raft_consensus.cc:2781] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:56.595625 32364 raft_consensus.cc:482] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:56.595659 32364 raft_consensus.cc:3037] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.595767 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8731e59b86f44ecaa2a0d5a955244cb" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:56.595880 26930 raft_consensus.cc:2454] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:56.596455 32364 raft_consensus.cc:504] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.596729 32364 leader_election.cc:288] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:56.597002 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8731e59b86f44ecaa2a0d5a955244cb" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:56.597098 29079 raft_consensus.cc:3037] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.597119 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8731e59b86f44ecaa2a0d5a955244cb" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:56.597215 26930 raft_consensus.cc:3037] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.597795 29079 raft_consensus.cc:2454] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:56.598064 26930 raft_consensus.cc:2454] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:56.598114 29283 leader_election.cc:302] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:56.598618 32364 raft_consensus.cc:2781] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:56.598703 32364 raft_consensus.cc:686] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:56.598817 32364 consensus_queue.cc:237] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:56.600679 26514 catalog_manager.cc:5484] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:56.613564 26930 consensus_queue.cc:237] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.614778 27064 raft_consensus.cc:1261] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.614908 26793 raft_consensus.cc:1261] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:56.614938 26862 consensus_peers.cc:485] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a528d9bbfa2488e8ce1e7b99b9e4d49. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.615141 1320 consensus_queue.cc:1035] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.615298 319 consensus_queue.cc:1035] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.616942 1320 raft_consensus.cc:2932] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.617301 27064 raft_consensus.cc:2932] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.619625 26514 catalog_manager.cc:5484] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.619998 26793 raft_consensus.cc:2932] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.638182 29079 consensus_queue.cc:237] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:56.638563 1299 raft_consensus.cc:482] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:56.638654 1299 raft_consensus.cc:504] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.639021 1299 leader_election.cc:288] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:56.639410 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca17309584740f28f6a9163a2c2198c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:56.639454 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca17309584740f28f6a9163a2c2198c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:56.639529 26930 raft_consensus.cc:2454] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:56.639569 29489 raft_consensus.cc:2454] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:58:56.639863 26661 raft_consensus.cc:1261] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.639914 29216 raft_consensus.cc:1261] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.640180 26998 leader_election.cc:302] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:56.640270 529 consensus_queue.cc:1035] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:56.640388 1299 raft_consensus.cc:2781] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:56.640442 1299 raft_consensus.cc:482] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:56.640477 1299 raft_consensus.cc:3037] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
W20240502 01:58:56.640520 29016 consensus_peers.cc:485] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0ea54806f7342e2bb3d2f287bb84bdc. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.641211 1299 raft_consensus.cc:504] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.641525 1299 leader_election.cc:288] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:56.641794 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca17309584740f28f6a9163a2c2198c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:56.641834 969 consensus_queue.cc:1035] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:56.641887 26930 raft_consensus.cc:3037] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.642797 26930 raft_consensus.cc:2454] T 7ca17309584740f28f6a9163a2c2198c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:56.643102 26998 leader_election.cc:302] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:56.643133 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ca17309584740f28f6a9163a2c2198c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:56.643229 29489 raft_consensus.cc:3037] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:56.643324 1299 raft_consensus.cc:2781] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:56.643400 1299 raft_consensus.cc:686] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:58:56.643520 1299 consensus_queue.cc:237] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.643702 529 raft_consensus.cc:2932] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.644194 29489 raft_consensus.cc:2454] T 7ca17309584740f28f6a9163a2c2198c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:58:56.644150 26661 raft_consensus.cc:2932] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.646154 29216 raft_consensus.cc:2932] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.646351 26512 catalog_manager.cc:5484] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:56.646705 26514 catalog_manager.cc:5484] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.665534 27064 consensus_queue.cc:237] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:56.666774 26930 raft_consensus.cc:1261] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.666775 26793 raft_consensus.cc:1261] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:56.667130 26995 consensus_peers.cc:485] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d53d3130fa84270ad934353a5dbab41. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.667236 32192 consensus_queue.cc:1035] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.667411 1299 consensus_queue.cc:1035] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.669059 32192 raft_consensus.cc:2932] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.669435 26793 raft_consensus.cc:2932] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.670912 26930 raft_consensus.cc:2932] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.671581 26514 catalog_manager.cc:5484] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.690260 29079 consensus_queue.cc:237] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:56.691519 26661 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.691589 29489 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:58:56.691723 29015 consensus_peers.cc:485] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a326a7899c064c95b426aae61d65208f. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.691972 1342 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:56.692158 1105 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:56.693810 1105 raft_consensus.cc:2932] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:56.694236 26661 raft_consensus.cc:2932] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:56.696871 26512 catalog_manager.cc:5484] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:56.698359 29489 raft_consensus.cc:2932] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:56.708801 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:56.712157 29079 raft_consensus.cc:1261] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.712185 26930 raft_consensus.cc:1261] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.712532 765 consensus_queue.cc:1035] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.712714 32364 consensus_queue.cc:1035] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.715188 26512 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet e8731e59b86f44ecaa2a0d5a955244cb (table rebalancer_dynamic_table [id=81041992874547a38070159676060e8b]). Expected version 1 got 0
I20240502 01:58:56.715394 832 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:56.715691 832 tablet.cc:1620] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:56.716058 26513 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet e8731e59b86f44ecaa2a0d5a955244cb (table rebalancer_dynamic_table [id=81041992874547a38070159676060e8b]). Expected version 1 got 0
I20240502 01:58:56.717962 1036 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:56.718263 1036 tablet.cc:1620] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:56.718339 26513 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet e8731e59b86f44ecaa2a0d5a955244cb (table rebalancer_dynamic_table [id=81041992874547a38070159676060e8b]). Expected version 1 got 0
I20240502 01:58:56.718652 1398 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:56.718940 1398 tablet.cc:1620] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:56.722553 27064 consensus_queue.cc:237] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.722352 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=81041992874547a38070159676060e8b] alter complete (version 1)
I20240502 01:58:56.723888 29216 raft_consensus.cc:1261] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:56.723987 26930 raft_consensus.cc:1261] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:58:56.724215 26998 consensus_peers.cc:485] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c46a7ee366464769b8498c4208ce8e9b. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.724277 32192 consensus_queue.cc:1035] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:56.724423 32573 consensus_queue.cc:1035] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:56.726274 32573 raft_consensus.cc:2932] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.727649 26930 raft_consensus.cc:2932] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.729490 29216 raft_consensus.cc:2932] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.729822 26513 catalog_manager.cc:5484] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.746464 26930 consensus_queue.cc:237] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:56.747794 27064 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.747802 26793 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:56.747817 26861 consensus_peers.cc:485] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 626774246b9445bc838427358e08f2cd. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.748296 319 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.748602 319 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.750403 319 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:56.752005 27064 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:56.752151 26793 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:56.755636 26513 catalog_manager.cc:5484] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:56.772996 27063 consensus_queue.cc:237] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:56.774163 26930 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.774300 26661 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.774575 32192 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.774740 1299 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:58:56.775024 26995 consensus_peers.cc:485] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0d96b487a964b90b720d8977a96fc5b. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.776517 32192 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.776918 26930 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.778870 26512 catalog_manager.cc:5484] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.779959 26661 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:56.797226 26930 consensus_queue.cc:237] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:56.798609 27063 raft_consensus.cc:1261] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:58:56.798635 26793 raft_consensus.cc:1261] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:58:56.798820 26862 consensus_peers.cc:485] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c150446ded240a2946d5afa10ecd1df. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:56.799085 1421 consensus_queue.cc:1035] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.799255 1048 consensus_queue.cc:1035] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:58:56.800945 1421 raft_consensus.cc:2932] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.801368 26793 raft_consensus.cc:2932] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.802628 27063 raft_consensus.cc:2932] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.804067 26514 catalog_manager.cc:5484] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:56.823900 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:56.829319 832 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:56.829628 832 tablet.cc:1620] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:56.830482 1398 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:56.830730 1398 tablet.cc:1620] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:56.832716 1036 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:56.833011 1036 tablet.cc:1620] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:56.835297 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=81041992874547a38070159676060e8b] alter complete (version 2)
I20240502 01:58:56.899219 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.900606 26640 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:56.900771 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:56.907099 29468 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:56.913419 29330 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:56.920931 1217 consensus_queue.cc:1035] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.921809 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:56.927162 31766 consensus_queue.cc:1035] T d2ef5e17842d47ccac3b908e9b389384 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:56.938318 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:56.944249 832 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:56.945045 1398 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:56.947631 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:56.948978 1036 tablet.cc:1722] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:56.951095 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=81041992874547a38070159676060e8b] alter complete (version 3)
I20240502 01:58:56.952032 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.002180 1527 ts_tablet_manager.cc:902] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:57.002673 1527 tablet_copy_client.cc:308] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:57.003135 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet 47f84b6ffb074bc682aa7b087b8c3614 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:56105)
I20240502 01:58:57.003222 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet 47f84b6ffb074bc682aa7b087b8c3614 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:56105: session id = 468dd27456c64d67b22e8a58af93d93d-47f84b6ffb074bc682aa7b087b8c3614
I20240502 01:58:57.003993 27083 tablet_copy_source_session.cc:205] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.004819 1527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f84b6ffb074bc682aa7b087b8c3614. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.006842 1527 tablet_copy_client.cc:783] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.007082 1527 tablet_copy_client.cc:647] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.008155 1527 tablet_copy_client.cc:523] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.009848 1527 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.033035 1527 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.033325 1527 tablet_bootstrap.cc:492] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:57.033499 1527 ts_tablet_manager.cc:1367] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240502 01:58:57.033888 1527 raft_consensus.cc:348] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.033995 1527 raft_consensus.cc:729] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.034174 1527 consensus_queue.cc:260] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.035179 1527 ts_tablet_manager.cc:1398] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240502 01:58:57.036451 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-47f84b6ffb074bc682aa7b087b8c3614 received from {username='slave'} at 127.19.43.199:56105
I20240502 01:58:57.036531 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-47f84b6ffb074bc682aa7b087b8c3614 on tablet 47f84b6ffb074bc682aa7b087b8c3614 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:57.052626 31766 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.057006 1217 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.059412 26506 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:57.059543 26506 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:57.062072 26506 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet e8731e59b86f44ecaa2a0d5a955244cb
I20240502 01:58:57.062698 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet e8731e59b86f44ecaa2a0d5a955244cb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:57 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:57.063055 1566 tablet_replica.cc:330] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:57.063124 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet e8731e59b86f44ecaa2a0d5a955244cb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:57 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:57.063247 1566 raft_consensus.cc:2229] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:57.063366 1566 raft_consensus.cc:2258] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.063678 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet e8731e59b86f44ecaa2a0d5a955244cb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:57 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:57.063900 1566 ts_tablet_manager.cc:1875] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:57.064019 1567 tablet_replica.cc:330] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:57.064352 1567 raft_consensus.cc:2229] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:57.065513 1567 raft_consensus.cc:2258] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.066100 1567 ts_tablet_manager.cc:1875] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:57.066464 1566 ts_tablet_manager.cc:1888] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:57.066543 1566 log.cc:1196] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/e8731e59b86f44ecaa2a0d5a955244cb
I20240502 01:58:57.066857 1566 ts_tablet_manager.cc:1909] T e8731e59b86f44ecaa2a0d5a955244cb P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:57.067039 1568 tablet_replica.cc:330] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:57.067219 1568 raft_consensus.cc:2229] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:57.067911 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet e8731e59b86f44ecaa2a0d5a955244cb (table rebalancer_dynamic_table_new_name [id=81041992874547a38070159676060e8b]) successfully deleted
I20240502 01:58:57.068444 1568 raft_consensus.cc:2258] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.068909 1567 ts_tablet_manager.cc:1888] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:57.068982 1567 log.cc:1196] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/e8731e59b86f44ecaa2a0d5a955244cb
I20240502 01:58:57.069275 1567 ts_tablet_manager.cc:1909] T e8731e59b86f44ecaa2a0d5a955244cb P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:57.069353 1568 ts_tablet_manager.cc:1875] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:57.070179 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet e8731e59b86f44ecaa2a0d5a955244cb (table rebalancer_dynamic_table_new_name [id=81041992874547a38070159676060e8b]) successfully deleted
I20240502 01:58:57.071904 1568 ts_tablet_manager.cc:1888] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:57.072022 1568 log.cc:1196] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/e8731e59b86f44ecaa2a0d5a955244cb
I20240502 01:58:57.072281 1568 ts_tablet_manager.cc:1909] T e8731e59b86f44ecaa2a0d5a955244cb P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:57.073045 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet e8731e59b86f44ecaa2a0d5a955244cb (table rebalancer_dynamic_table_new_name [id=81041992874547a38070159676060e8b]) successfully deleted
I20240502 01:58:57.082746 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:57.083310 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:57.084050 29330 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:57.085795 1299 consensus_queue.cc:1035] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.090066 26640 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:57.090231 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:57.098271 29468 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:57.112228 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:57.112677 1402 consensus_queue.cc:1035] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.125084 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.133035 1585 ts_tablet_manager.cc:902] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:57.133536 1585 tablet_copy_client.cc:308] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:57.133973 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet c46a7ee366464769b8498c4208ce8e9b from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:53349)
I20240502 01:58:57.134083 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet c46a7ee366464769b8498c4208ce8e9b from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:53349: session id = 7c693d0933c64d9babc07f404df45cd1-c46a7ee366464769b8498c4208ce8e9b
I20240502 01:58:57.134934 27083 tablet_copy_source_session.cc:205] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.135744 1585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46a7ee366464769b8498c4208ce8e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.138319 1585 tablet_copy_client.cc:783] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.138494 1585 tablet_copy_client.cc:647] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.142800 1581 ts_tablet_manager.cc:902] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:57.143352 1581 tablet_copy_client.cc:308] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:58:57.143816 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet a0ea54806f7342e2bb3d2f287bb84bdc from peer 68e5b1eeb04448fea818a14991eab87c ({username='slave'} at 127.19.43.196:54885)
I20240502 01:58:57.143919 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet a0ea54806f7342e2bb3d2f287bb84bdc from peer 68e5b1eeb04448fea818a14991eab87c at {username='slave'} at 127.19.43.196:54885: session id = 68e5b1eeb04448fea818a14991eab87c-a0ea54806f7342e2bb3d2f287bb84bdc
I20240502 01:58:57.144795 29102 tablet_copy_source_session.cc:205] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.146579 1581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ea54806f7342e2bb3d2f287bb84bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.146862 1585 tablet_copy_client.cc:523] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.148200 1585 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:57.149209 1581 tablet_copy_client.cc:783] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.149338 1581 tablet_copy_client.cc:647] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.150301 1581 tablet_copy_client.cc:523] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.151623 1581 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:57.155293 1585 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.155304 1581 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.155521 1585 tablet_bootstrap.cc:492] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:57.155668 1585 ts_tablet_manager.cc:1367] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20240502 01:58:57.155809 1581 tablet_bootstrap.cc:492] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: Bootstrap complete.
I20240502 01:58:57.156001 1581 ts_tablet_manager.cc:1367] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:58:57.156072 1585 raft_consensus.cc:348] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.156198 1585 raft_consensus.cc:729] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.156306 1585 consensus_queue.cc:260] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.156559 1585 ts_tablet_manager.cc:1398] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:57.156847 1581 raft_consensus.cc:348] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.157011 1581 raft_consensus.cc:729] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.157128 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-c46a7ee366464769b8498c4208ce8e9b received from {username='slave'} at 127.19.43.200:53349
I20240502 01:58:57.157182 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-c46a7ee366464769b8498c4208ce8e9b on tablet c46a7ee366464769b8498c4208ce8e9b with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:57.158298 1581 consensus_queue.cc:260] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.159225 1581 ts_tablet_manager.cc:1398] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:57.160441 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session 68e5b1eeb04448fea818a14991eab87c-a0ea54806f7342e2bb3d2f287bb84bdc received from {username='slave'} at 127.19.43.196:54885
I20240502 01:58:57.160511 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session 68e5b1eeb04448fea818a14991eab87c-a0ea54806f7342e2bb3d2f287bb84bdc on tablet a0ea54806f7342e2bb3d2f287bb84bdc with peer 68e5b1eeb04448fea818a14991eab87c
I20240502 01:58:57.163856 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:57.164297 1598 ts_tablet_manager.cc:902] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:57.166119 1598 tablet_copy_client.cc:308] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:58:57.166529 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet a326a7899c064c95b426aae61d65208f from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:41057)
I20240502 01:58:57.166612 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet a326a7899c064c95b426aae61d65208f from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:41057: session id = b290c1bf893444078ed721f25995d124-a326a7899c064c95b426aae61d65208f
I20240502 01:58:57.167376 29102 tablet_copy_source_session.cc:205] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.168148 1598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a326a7899c064c95b426aae61d65208f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.170461 1598 tablet_copy_client.cc:783] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.170583 1598 tablet_copy_client.cc:647] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.171284 26775 tablet_service.cc:1467] Processing CreateTablet for tablet f9f444a1a8a247faa2b1d7a8fe72062f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339]), partition=
I20240502 01:58:57.171433 29059 tablet_service.cc:1467] Processing CreateTablet for tablet f9f444a1a8a247faa2b1d7a8fe72062f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339]), partition=
I20240502 01:58:57.171501 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f444a1a8a247faa2b1d7a8fe72062f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.171633 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f444a1a8a247faa2b1d7a8fe72062f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.173569 1607 tablet_bootstrap.cc:492] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:57.174620 1598 tablet_copy_client.cc:523] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.174806 1608 tablet_bootstrap.cc:492] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:57.174898 29330 tablet_service.cc:1467] Processing CreateTablet for tablet f9f444a1a8a247faa2b1d7a8fe72062f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339]), partition=
I20240502 01:58:57.175127 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f444a1a8a247faa2b1d7a8fe72062f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.175705 1608 tablet_bootstrap.cc:654] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.175798 1598 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:57.176764 1608 tablet_bootstrap.cc:492] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:57.176837 1608 ts_tablet_manager.cc:1367] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.177253 1611 tablet_bootstrap.cc:492] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.177359 1608 raft_consensus.cc:348] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.177520 1608 raft_consensus.cc:374] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.177599 1608 raft_consensus.cc:729] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.177790 1608 consensus_queue.cc:260] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.178014 1607 tablet_bootstrap.cc:654] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.178213 1608 ts_tablet_manager.cc:1398] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.180043 1611 tablet_bootstrap.cc:654] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.181061 1598 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.181290 1598 tablet_bootstrap.cc:492] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:58:57.181408 1611 tablet_bootstrap.cc:492] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:57.181447 1598 ts_tablet_manager.cc:1367] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20240502 01:58:57.181494 1611 ts_tablet_manager.cc:1367] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:58:57.181849 1598 raft_consensus.cc:348] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:57.181885 1611 raft_consensus.cc:348] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.181963 1611 raft_consensus.cc:374] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.181959 1598 raft_consensus.cc:729] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.182051 1611 raft_consensus.cc:729] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.182199 1611 consensus_queue.cc:260] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.182382 1598 consensus_queue.cc:260] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:57.182663 1598 ts_tablet_manager.cc:1398] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.183295 1611 ts_tablet_manager.cc:1398] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.183984 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session b290c1bf893444078ed721f25995d124-a326a7899c064c95b426aae61d65208f received from {username='slave'} at 127.19.43.194:41057
I20240502 01:58:57.184049 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session b290c1bf893444078ed721f25995d124-a326a7899c064c95b426aae61d65208f on tablet a326a7899c064c95b426aae61d65208f with peer b290c1bf893444078ed721f25995d124
I20240502 01:58:57.184149 1607 tablet_bootstrap.cc:492] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:57.184237 1607 ts_tablet_manager.cc:1367] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20240502 01:58:57.184666 1607 raft_consensus.cc:348] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.184764 1607 raft_consensus.cc:374] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.185683 1607 raft_consensus.cc:729] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.185846 1607 consensus_queue.cc:260] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.186275 1607 ts_tablet_manager.cc:1398] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:57.189312 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:57.195331 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 1bf76f857ef845f7895c18ca7766051d (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.195607 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bf76f857ef845f7895c18ca7766051d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.195669 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 1bf76f857ef845f7895c18ca7766051d (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.195917 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bf76f857ef845f7895c18ca7766051d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.197656 1621 tablet_bootstrap.cc:492] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:57.198354 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 463056ff0e244d2fa7dd6c4c70f3eded (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.198586 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463056ff0e244d2fa7dd6c4c70f3eded. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.198513 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 089d9dc33a594712ac04ad28a0cd37b1 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.198757 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089d9dc33a594712ac04ad28a0cd37b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.198716 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 089d9dc33a594712ac04ad28a0cd37b1 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.198926 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089d9dc33a594712ac04ad28a0cd37b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.200716 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 089d9dc33a594712ac04ad28a0cd37b1 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.200883 1608 tablet_bootstrap.cc:492] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:57.200956 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089d9dc33a594712ac04ad28a0cd37b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.201110 1622 tablet_bootstrap.cc:492] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:57.201689 1608 tablet_bootstrap.cc:654] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.201962 1622 tablet_bootstrap.cc:654] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.198513 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 463056ff0e244d2fa7dd6c4c70f3eded (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.202472 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463056ff0e244d2fa7dd6c4c70f3eded. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.203493 1622 tablet_bootstrap.cc:492] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:57.203567 1622 ts_tablet_manager.cc:1367] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:57.203927 1611 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.203956 1622 raft_consensus.cc:348] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.204074 1622 raft_consensus.cc:374] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.204105 1622 raft_consensus.cc:729] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.204218 1622 consensus_queue.cc:260] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.204469 1622 ts_tablet_manager.cc:1398] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.204751 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 463056ff0e244d2fa7dd6c4c70f3eded (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.204838 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 1bf76f857ef845f7895c18ca7766051d (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.204946 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463056ff0e244d2fa7dd6c4c70f3eded. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.205097 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bf76f857ef845f7895c18ca7766051d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.205219 1611 tablet_bootstrap.cc:654] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.206605 1611 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:57.206681 1611 ts_tablet_manager.cc:1367] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:58:57.207068 1611 raft_consensus.cc:348] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.207190 1611 raft_consensus.cc:374] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.207222 1611 raft_consensus.cc:729] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.207335 1611 consensus_queue.cc:260] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.207440 1622 tablet_bootstrap.cc:492] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:57.207628 1611 ts_tablet_manager.cc:1398] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:57.208248 1622 tablet_bootstrap.cc:654] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.208442 1585 ts_tablet_manager.cc:902] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:57.208845 1585 tablet_copy_client.cc:308] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:57.209273 1622 tablet_bootstrap.cc:492] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:57.209347 1622 ts_tablet_manager.cc:1367] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.209468 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:45861)
I20240502 01:58:57.209580 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:45861: session id = 7c693d0933c64d9babc07f404df45cd1-4a528d9bbfa2488e8ce1e7b99b9e4d49
I20240502 01:58:57.209710 1622 raft_consensus.cc:348] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.209815 1622 raft_consensus.cc:374] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.209861 1622 raft_consensus.cc:729] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.209975 1622 consensus_queue.cc:260] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.210253 1622 ts_tablet_manager.cc:1398] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.210507 1625 ts_tablet_manager.cc:902] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:57.210565 26950 tablet_copy_source_session.cc:205] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.210973 1625 tablet_copy_client.cc:308] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:57.211483 1585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.212570 1637 tablet_bootstrap.cc:492] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:58:57.212661 1621 tablet_bootstrap.cc:654] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.213430 1585 tablet_copy_client.cc:783] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.213505 1637 tablet_bootstrap.cc:654] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.213557 1585 tablet_copy_client.cc:647] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.213589 1623 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:57.214077 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 626774246b9445bc838427358e08f2cd from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:51525)
I20240502 01:58:57.214164 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 626774246b9445bc838427358e08f2cd from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:51525: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-626774246b9445bc838427358e08f2cd
I20240502 01:58:57.214571 1637 tablet_bootstrap.cc:492] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:58:57.214645 1637 ts_tablet_manager.cc:1367] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:57.214649 1623 tablet_bootstrap.cc:654] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.215075 1637 raft_consensus.cc:348] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.215255 1637 raft_consensus.cc:374] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.215337 1637 raft_consensus.cc:729] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.215574 1637 consensus_queue.cc:260] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.215682 26950 tablet_copy_source_session.cc:205] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.216771 1625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626774246b9445bc838427358e08f2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.216883 1585 tablet_copy_client.cc:523] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.217361 1608 tablet_bootstrap.cc:492] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:57.217438 1608 ts_tablet_manager.cc:1367] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 01:58:57.217855 1608 raft_consensus.cc:348] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.217949 1608 raft_consensus.cc:374] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.217979 1608 raft_consensus.cc:729] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.217995 1585 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:57.218123 1608 consensus_queue.cc:260] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.218415 1608 ts_tablet_manager.cc:1398] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.218571 1608 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:57.218781 1625 tablet_copy_client.cc:783] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.218897 1625 tablet_copy_client.cc:647] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.219400 1637 ts_tablet_manager.cc:1398] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:57.219549 1608 tablet_bootstrap.cc:654] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.220922 1625 tablet_copy_client.cc:523] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.221390 1621 tablet_bootstrap.cc:492] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:57.221470 1621 ts_tablet_manager.cc:1367] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20240502 01:58:57.221509 1608 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:57.221572 1608 ts_tablet_manager.cc:1367] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:57.221920 1608 raft_consensus.cc:348] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.221915 1621 raft_consensus.cc:348] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.222010 1608 raft_consensus.cc:374] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.222010 1621 raft_consensus.cc:374] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.222067 1608 raft_consensus.cc:729] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.222080 1621 raft_consensus.cc:729] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.222147 1625 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:57.222173 1608 consensus_queue.cc:260] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.222260 1621 consensus_queue.cc:260] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.222432 1608 ts_tablet_manager.cc:1398] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.222590 1621 ts_tablet_manager.cc:1398] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.225662 1623 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:57.225785 1623 ts_tablet_manager.cc:1367] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:58:57.226395 1623 raft_consensus.cc:348] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.226651 1623 raft_consensus.cc:374] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.226737 1623 raft_consensus.cc:729] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.226918 1623 consensus_queue.cc:260] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.228267 1623 ts_tablet_manager.cc:1398] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.228885 1623 tablet_bootstrap.cc:492] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:57.229735 1623 tablet_bootstrap.cc:654] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.230764 1623 tablet_bootstrap.cc:492] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:57.230840 1623 ts_tablet_manager.cc:1367] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:57.231230 1623 raft_consensus.cc:348] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.231323 1623 raft_consensus.cc:374] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.231349 1623 raft_consensus.cc:729] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.231443 1623 consensus_queue.cc:260] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.231671 1623 ts_tablet_manager.cc:1398] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.248014 1625 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.248270 1625 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:58:57.248468 1625 ts_tablet_manager.cc:1367] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.027s user 0.020s sys 0.005s
I20240502 01:58:57.248922 1625 raft_consensus.cc:348] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:57.249061 1625 raft_consensus.cc:729] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.249253 1625 consensus_queue.cc:260] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:57.250196 1625 ts_tablet_manager.cc:1398] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.250633 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-626774246b9445bc838427358e08f2cd received from {username='slave'} at 127.19.43.198:51525
I20240502 01:58:57.250708 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-626774246b9445bc838427358e08f2cd on tablet 626774246b9445bc838427358e08f2cd with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:57.252280 1585 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.252983 1585 tablet_bootstrap.cc:492] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:57.253188 1585 ts_tablet_manager.cc:1367] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.000s
I20240502 01:58:57.253846 1585 raft_consensus.cc:348] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.254948 1585 raft_consensus.cc:729] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.255115 1585 consensus_queue.cc:260] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.255458 1585 ts_tablet_manager.cc:1398] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.255874 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-4a528d9bbfa2488e8ce1e7b99b9e4d49 received from {username='slave'} at 127.19.43.200:45861
I20240502 01:58:57.255942 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-4a528d9bbfa2488e8ce1e7b99b9e4d49 on tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:57.269188 1527 ts_tablet_manager.cc:902] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:57.269650 1527 tablet_copy_client.cc:308] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:57.270151 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet 3d53d3130fa84270ad934353a5dbab41 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:56105)
I20240502 01:58:57.270241 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet 3d53d3130fa84270ad934353a5dbab41 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:56105: session id = 468dd27456c64d67b22e8a58af93d93d-3d53d3130fa84270ad934353a5dbab41
I20240502 01:58:57.271158 27083 tablet_copy_source_session.cc:205] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.272019 1527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d53d3130fa84270ad934353a5dbab41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.272765 1679 ts_tablet_manager.cc:902] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:57.273236 1679 tablet_copy_client.cc:308] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:58:57.273615 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet a0d96b487a964b90b720d8977a96fc5b from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:56105)
I20240502 01:58:57.273691 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet a0d96b487a964b90b720d8977a96fc5b from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:56105: session id = 468dd27456c64d67b22e8a58af93d93d-a0d96b487a964b90b720d8977a96fc5b
I20240502 01:58:57.274405 27083 tablet_copy_source_session.cc:205] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.275944 1527 tablet_copy_client.cc:783] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.276402 1527 tablet_copy_client.cc:647] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.276708 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d96b487a964b90b720d8977a96fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.277449 1527 tablet_copy_client.cc:523] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.278652 1527 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.278894 1679 tablet_copy_client.cc:783] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.279089 1679 tablet_copy_client.cc:647] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.280697 1679 tablet_copy_client.cc:523] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.282150 1679 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.302491 1585 ts_tablet_manager.cc:902] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:57.302892 1585 tablet_copy_client.cc:308] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:57.303427 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 2c150446ded240a2946d5afa10ecd1df from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:45861)
I20240502 01:58:57.303526 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 2c150446ded240a2946d5afa10ecd1df from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:45861: session id = 7c693d0933c64d9babc07f404df45cd1-2c150446ded240a2946d5afa10ecd1df
I20240502 01:58:57.304386 26950 tablet_copy_source_session.cc:205] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:57.305150 1585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c150446ded240a2946d5afa10ecd1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.307384 1585 tablet_copy_client.cc:783] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:57.307507 1585 tablet_copy_client.cc:647] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:57.311883 1527 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.312125 1527 tablet_bootstrap.cc:492] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:57.312274 1527 ts_tablet_manager.cc:1367] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.004s
I20240502 01:58:57.312661 1527 raft_consensus.cc:348] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.312775 1527 raft_consensus.cc:729] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.312935 1527 consensus_queue.cc:260] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.313231 1527 ts_tablet_manager.cc:1398] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:58:57.313377 1679 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.313733 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-3d53d3130fa84270ad934353a5dbab41 received from {username='slave'} at 127.19.43.199:56105
I20240502 01:58:57.313804 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-3d53d3130fa84270ad934353a5dbab41 on tablet 3d53d3130fa84270ad934353a5dbab41 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:57.313838 1679 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:58:57.313899 1595 raft_consensus.cc:482] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:57.313982 1679 ts_tablet_manager.cc:1367] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.000s
I20240502 01:58:57.313988 1595 raft_consensus.cc:504] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.314368 1595 leader_election.cc:288] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:57.314375 1679 raft_consensus.cc:348] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.314468 1679 raft_consensus.cc:729] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.314600 1679 consensus_queue.cc:260] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.314914 1679 ts_tablet_manager.cc:1398] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.314962 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:57.315088 29079 raft_consensus.cc:2454] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:57.315302 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:57.315410 29216 raft_consensus.cc:2454] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:57.315588 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-a0d96b487a964b90b720d8977a96fc5b received from {username='slave'} at 127.19.43.199:56105
I20240502 01:58:57.315651 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-a0d96b487a964b90b720d8977a96fc5b on tablet a0d96b487a964b90b720d8977a96fc5b with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:58:57.315680 29421 leader_election.cc:302] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:57.315878 1595 raft_consensus.cc:2781] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:57.315930 1595 raft_consensus.cc:482] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:57.315968 1595 raft_consensus.cc:3037] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.316833 1595 raft_consensus.cc:504] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.317153 1595 leader_election.cc:288] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:57.317672 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:57.317766 29216 raft_consensus.cc:3037] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.318066 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:57.318179 29079 raft_consensus.cc:3037] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.318778 29216 raft_consensus.cc:2454] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:57.319092 1585 tablet_copy_client.cc:523] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:57.319181 29079 raft_consensus.cc:2454] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:57.319195 29421 leader_election.cc:302] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:57.319334 32697 raft_consensus.cc:2781] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:57.319396 32697 raft_consensus.cc:686] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:57.319500 32697 consensus_queue.cc:237] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.320242 1585 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:57.323590 26512 catalog_manager.cc:5484] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:57.340785 1585 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:57.341027 1585 tablet_bootstrap.cc:492] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:58:57.341192 1585 ts_tablet_manager.cc:1367] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.000s
I20240502 01:58:57.341557 1585 raft_consensus.cc:348] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.341645 1585 raft_consensus.cc:729] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.341760 1585 consensus_queue.cc:260] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.342010 1585 ts_tablet_manager.cc:1398] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:57.342562 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-2c150446ded240a2946d5afa10ecd1df received from {username='slave'} at 127.19.43.200:45861
I20240502 01:58:57.342638 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-2c150446ded240a2946d5afa10ecd1df on tablet 2c150446ded240a2946d5afa10ecd1df with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:57.353089 1217 raft_consensus.cc:482] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:57.353178 1217 raft_consensus.cc:504] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.353556 1217 leader_election.cc:288] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:57.353853 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bf76f857ef845f7895c18ca7766051d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:57.353868 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bf76f857ef845f7895c18ca7766051d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:58:57.353964 29216 raft_consensus.cc:2454] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:57.353966 27063 raft_consensus.cc:2454] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:57.354225 26594 leader_election.cc:302] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:57.354375 1217 raft_consensus.cc:2781] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:57.354429 1217 raft_consensus.cc:482] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:57.354454 1217 raft_consensus.cc:3037] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.355142 1217 raft_consensus.cc:504] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.355459 1217 leader_election.cc:288] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:58:57.355762 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bf76f857ef845f7895c18ca7766051d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:58:57.355764 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bf76f857ef845f7895c18ca7766051d" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:57.355857 27063 raft_consensus.cc:3037] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.355859 29216 raft_consensus.cc:3037] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.356511 29216 raft_consensus.cc:2454] T 1bf76f857ef845f7895c18ca7766051d P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:57.356537 27063 raft_consensus.cc:2454] T 1bf76f857ef845f7895c18ca7766051d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:57.356782 26594 leader_election.cc:302] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:57.356959 1217 raft_consensus.cc:2781] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:57.357053 1217 raft_consensus.cc:686] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:57.357174 1217 consensus_queue.cc:237] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:57.358942 26513 catalog_manager.cc:5484] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:57.388429 948 raft_consensus.cc:482] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:57.388525 948 raft_consensus.cc:504] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.388866 948 leader_election.cc:288] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:57.389163 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f444a1a8a247faa2b1d7a8fe72062f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:57.389267 29079 raft_consensus.cc:2454] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:57.389451 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f444a1a8a247faa2b1d7a8fe72062f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:57.389552 29350 raft_consensus.cc:2454] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:58:57.389823 26727 leader_election.cc:302] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:57.389981 948 raft_consensus.cc:2781] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:57.390064 948 raft_consensus.cc:482] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:57.390100 948 raft_consensus.cc:3037] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.390796 948 raft_consensus.cc:504] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.391052 948 leader_election.cc:288] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:57.391351 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f444a1a8a247faa2b1d7a8fe72062f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:57.391355 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f444a1a8a247faa2b1d7a8fe72062f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:57.391444 29079 raft_consensus.cc:3037] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.391464 29350 raft_consensus.cc:3037] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.392123 29079 raft_consensus.cc:2454] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:57.392313 29350 raft_consensus.cc:2454] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:58:57.392511 26727 leader_election.cc:302] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:57.392753 948 raft_consensus.cc:2781] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:57.392927 948 raft_consensus.cc:686] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:58:57.393056 948 consensus_queue.cc:237] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:57.394937 26513 catalog_manager.cc:5484] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:57.432608 32364 raft_consensus.cc:482] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:57.432700 32364 raft_consensus.cc:504] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.433056 32364 leader_election.cc:288] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:57.433357 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "089d9dc33a594712ac04ad28a0cd37b1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:57.433358 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "089d9dc33a594712ac04ad28a0cd37b1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:57.433480 29079 raft_consensus.cc:2454] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:57.433480 29487 raft_consensus.cc:2454] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:57.433708 29286 leader_election.cc:302] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:57.433873 32364 raft_consensus.cc:2781] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:57.433933 32364 raft_consensus.cc:482] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:57.433970 32364 raft_consensus.cc:3037] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.434682 32364 raft_consensus.cc:504] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.434983 32364 leader_election.cc:288] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:57.435257 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "089d9dc33a594712ac04ad28a0cd37b1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:57.435257 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "089d9dc33a594712ac04ad28a0cd37b1" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:57.435366 29079 raft_consensus.cc:3037] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.435366 29487 raft_consensus.cc:3037] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:57.436249 29079 raft_consensus.cc:2454] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:57.436254 29487 raft_consensus.cc:2454] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:57.436518 29286 leader_election.cc:302] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:57.436676 32364 raft_consensus.cc:2781] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:57.436753 32364 raft_consensus.cc:686] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:57.436889 32364 consensus_queue.cc:237] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.438643 26513 catalog_manager.cc:5484] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:57.459787 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:57.463855 29330 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:57.464722 29468 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:57.471741 29058 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.473542 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:57.475140 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:57.490252 26775 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.492048 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:57.520517 27063 raft_consensus.cc:1203] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:57.530320 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:57.534065 29350 raft_consensus.cc:1261] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:57.534106 29079 raft_consensus.cc:1261] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:57.534488 1693 consensus_queue.cc:1035] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.534713 948 consensus_queue.cc:1035] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.536446 29350 raft_consensus.cc:1203] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:57.537534 26514 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet f9f444a1a8a247faa2b1d7a8fe72062f (table rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339]). Expected version 1 got 0
I20240502 01:58:57.537842 26513 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet f9f444a1a8a247faa2b1d7a8fe72062f (table rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339]). Expected version 1 got 0
I20240502 01:58:57.538442 1722 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:57.539746 1723 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:57.540095 1723 tablet.cc:1620] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:57.543495 1722 tablet.cc:1620] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:57.545240 26514 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339] alter complete (version 1)
I20240502 01:58:57.549535 26514 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet f9f444a1a8a247faa2b1d7a8fe72062f (table rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339]). Expected version 1 got 0
I20240502 01:58:57.552294 1724 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:57.552553 1724 tablet.cc:1620] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:57.603667 26793 raft_consensus.cc:1203] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:57.611971 29487 raft_consensus.cc:1203] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:58:57.623521 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:57.628618 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:57.630012 26775 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.638185 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:57.639462 29058 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.642402 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:57.642500 29468 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:57.644145 29330 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:57.653257 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:57.667097 1722 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:57.667425 1722 tablet.cc:1620] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:57.668828 1724 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:57.669112 1724 tablet.cc:1620] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:57.673698 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=add4d31490fb4e13afed056186ab9339] alter complete (version 2)
I20240502 01:58:57.675062 1723 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:57.675391 1723 tablet.cc:1620] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:57.689432 29349 raft_consensus.cc:1203] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:57.709918 1595 consensus_queue.cc:1035] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.712872 32697 consensus_queue.cc:1035] T 463056ff0e244d2fa7dd6c4c70f3eded P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.715128 29487 raft_consensus.cc:1203] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:57.715216 29216 raft_consensus.cc:1203] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:57.722756 27063 consensus_queue.cc:237] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:57.723802 29487 raft_consensus.cc:1261] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:57.723891 26930 raft_consensus.cc:1261] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:57.724269 1299 consensus_queue.cc:1035] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:57.724418 1402 consensus_queue.cc:1035] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:57.725836 1402 raft_consensus.cc:2932] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:57.727815 26927 raft_consensus.cc:2932] T c46a7ee366464769b8498c4208ce8e9b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:57.729979 26512 catalog_manager.cc:5484] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:57.731014 29487 raft_consensus.cc:2932] T c46a7ee366464769b8498c4208ce8e9b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:57.732858 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet c46a7ee366464769b8498c4208ce8e9b with delete_type TABLET_DATA_TOMBSTONED (TS 0d83fdd1e6e04cf5a5602994d1dbc0df not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:57.733261 1794 tablet_replica.cc:330] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:57.733542 1794 raft_consensus.cc:2229] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:57.733731 1794 raft_consensus.cc:2258] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.734239 1794 ts_tablet_manager.cc:1875] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:57.736191 1794 ts_tablet_manager.cc:1888] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:57.736258 1794 log.cc:1196] T c46a7ee366464769b8498c4208ce8e9b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/c46a7ee366464769b8498c4208ce8e9b
I20240502 01:58:57.736685 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet c46a7ee366464769b8498c4208ce8e9b (table rebalancer_extra_table_0 [id=682bab6cacdb4110a2125888f67c7c84]) successfully deleted
I20240502 01:58:57.757124 29487 raft_consensus.cc:1203] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:57.778849 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:57.784348 1722 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:57.784667 1723 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:57.789441 1724 tablet.cc:1722] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:57.790334 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=add4d31490fb4e13afed056186ab9339] alter complete (version 3)
I20240502 01:58:57.827236 29330 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:57.831140 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:57.834297 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:57.839169 29468 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:57.840512 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:57.850185 29058 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.861971 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:57.862777 26775 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:57.870771 32364 consensus_queue.cc:1035] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.871181 1217 consensus_queue.cc:1035] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.873023 29349 raft_consensus.cc:1203] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:58:57.881631 765 consensus_queue.cc:1035] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.892678 26512 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:57.892807 26512 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:57.894712 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet f9f444a1a8a247faa2b1d7a8fe72062f
I20240502 01:58:57.895412 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f444a1a8a247faa2b1d7a8fe72062f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:57 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:57.895641 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f444a1a8a247faa2b1d7a8fe72062f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:57 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:57.895848 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f444a1a8a247faa2b1d7a8fe72062f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:57 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:58:57.897181 1845 tablet_replica.cc:330] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:57.897325 1845 raft_consensus.cc:2229] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:57.897508 1846 tablet_replica.cc:330] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:57.897672 1845 raft_consensus.cc:2258] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.898386 1846 raft_consensus.cc:2229] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:57.898495 31766 consensus_queue.cc:1035] T 1bf76f857ef845f7895c18ca7766051d P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:57.898710 1846 raft_consensus.cc:2258] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.900391 1845 ts_tablet_manager.cc:1875] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:57.900804 1846 ts_tablet_manager.cc:1875] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:57.902016 1847 tablet_replica.cc:330] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:58:57.902223 1847 raft_consensus.cc:2229] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:57.903003 1847 raft_consensus.cc:2258] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:57.903170 1846 ts_tablet_manager.cc:1888] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:57.903309 1846 log.cc:1196] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/f9f444a1a8a247faa2b1d7a8fe72062f
I20240502 01:58:57.903743 1846 ts_tablet_manager.cc:1909] T f9f444a1a8a247faa2b1d7a8fe72062f P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:58:57.903802 1845 ts_tablet_manager.cc:1888] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:57.903878 1845 log.cc:1196] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/f9f444a1a8a247faa2b1d7a8fe72062f
I20240502 01:58:57.905081 1847 ts_tablet_manager.cc:1875] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:57.905413 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet f9f444a1a8a247faa2b1d7a8fe72062f (table rebalancer_dynamic_table_new_name [id=add4d31490fb4e13afed056186ab9339]) successfully deleted
I20240502 01:58:57.907279 1847 ts_tablet_manager.cc:1888] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:57.907348 1847 log.cc:1196] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/f9f444a1a8a247faa2b1d7a8fe72062f
I20240502 01:58:57.907435 1845 ts_tablet_manager.cc:1909] T f9f444a1a8a247faa2b1d7a8fe72062f P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:57.908183 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet f9f444a1a8a247faa2b1d7a8fe72062f (table rebalancer_dynamic_table_new_name [id=add4d31490fb4e13afed056186ab9339]) successfully deleted
I20240502 01:58:57.908244 1847 ts_tablet_manager.cc:1909] T f9f444a1a8a247faa2b1d7a8fe72062f P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:58:57.908783 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet f9f444a1a8a247faa2b1d7a8fe72062f (table rebalancer_dynamic_table_new_name [id=add4d31490fb4e13afed056186ab9339]) successfully deleted
I20240502 01:58:57.916429 29079 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a326a7899c064c95b426aae61d65208f"
dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50822
I20240502 01:58:57.916568 29079 raft_consensus.cc:595] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Received request to transfer leadership
I20240502 01:58:57.966380 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:57.973603 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 12420fbcddc5446b857d2e6a01f40aaa (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.973901 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12420fbcddc5446b857d2e6a01f40aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.974014 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 12420fbcddc5446b857d2e6a01f40aaa (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.974280 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12420fbcddc5446b857d2e6a01f40aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.975916 29468 tablet_service.cc:1467] Processing CreateTablet for tablet b98e2f3fb2ea4241a16db8b4bb60b4b8 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.976187 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98e2f3fb2ea4241a16db8b4bb60b4b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.976428 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 12420fbcddc5446b857d2e6a01f40aaa (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.976650 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12420fbcddc5446b857d2e6a01f40aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.976626 29195 tablet_service.cc:1467] Processing CreateTablet for tablet b98e2f3fb2ea4241a16db8b4bb60b4b8 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.976871 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98e2f3fb2ea4241a16db8b4bb60b4b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.976910 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 499d2ca7bffc42748591db5280f829f2 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.977090 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499d2ca7bffc42748591db5280f829f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.978812 1894 tablet_bootstrap.cc:492] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:57.979259 1895 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.979908 1894 tablet_bootstrap.cc:654] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.980119 1895 tablet_bootstrap.cc:654] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.980369 1892 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:57.981029 1894 tablet_bootstrap.cc:492] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:57.981029 1895 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:57.981107 1895 ts_tablet_manager.cc:1367] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.981107 1894 ts_tablet_manager.cc:1367] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:57.981446 1892 tablet_bootstrap.cc:654] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.981495 1895 raft_consensus.cc:348] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.981495 1894 raft_consensus.cc:348] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:57.981590 1895 raft_consensus.cc:374] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.981590 1894 raft_consensus.cc:374] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.981619 1894 raft_consensus.cc:729] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.981729 1894 consensus_queue.cc:260] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:57.981817 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 499d2ca7bffc42748591db5280f829f2 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.982044 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499d2ca7bffc42748591db5280f829f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.982245 1896 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:57.981997 1894 ts_tablet_manager.cc:1398] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:57.985337 1896 tablet_bootstrap.cc:654] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.986542 1892 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:57.986683 1892 ts_tablet_manager.cc:1367] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 01:58:57.981619 1895 raft_consensus.cc:729] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.987378 1895 consensus_queue.cc:260] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.987402 29330 tablet_service.cc:1467] Processing CreateTablet for tablet b98e2f3fb2ea4241a16db8b4bb60b4b8 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.987685 1895 ts_tablet_manager.cc:1398] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:57.987847 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98e2f3fb2ea4241a16db8b4bb60b4b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.988534 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 499d2ca7bffc42748591db5280f829f2 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:57.988739 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499d2ca7bffc42748591db5280f829f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:57.989814 1895 tablet_bootstrap.cc:492] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:57.990597 1895 tablet_bootstrap.cc:654] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.991274 1892 raft_consensus.cc:348] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.991449 1892 raft_consensus.cc:374] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.991609 1892 raft_consensus.cc:729] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.991818 1892 consensus_queue.cc:260] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.992249 1892 ts_tablet_manager.cc:1398] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:58:57.993105 1896 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:57.993250 1896 ts_tablet_manager.cc:1367] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:58:57.993729 1896 raft_consensus.cc:348] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.993889 1896 raft_consensus.cc:374] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:57.993971 1896 raft_consensus.cc:729] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:57.994167 1896 consensus_queue.cc:260] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:57.995203 1896 ts_tablet_manager.cc:1398] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:57.995404 1896 tablet_bootstrap.cc:492] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:57.996311 1896 tablet_bootstrap.cc:654] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:57.996663 26514 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:58.001139 1895 tablet_bootstrap.cc:492] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:58.001220 1895 ts_tablet_manager.cc:1367] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:58:58.001628 1895 raft_consensus.cc:348] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.001722 1895 raft_consensus.cc:374] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.001751 1895 raft_consensus.cc:729] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.003567 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbc0190549348d29fb307282f3a7a47 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf]), partition=
I20240502 01:58:58.003578 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbc0190549348d29fb307282f3a7a47 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf]), partition=
I20240502 01:58:58.003733 765 raft_consensus.cc:482] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.003772 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbc0190549348d29fb307282f3a7a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.003814 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbc0190549348d29fb307282f3a7a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.003805 765 raft_consensus.cc:504] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.004154 765 leader_election.cc:288] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.004531 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12420fbcddc5446b857d2e6a01f40aaa" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:58.004642 26927 raft_consensus.cc:2454] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:58.005244 29286 leader_election.cc:302] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:58.005281 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12420fbcddc5446b857d2e6a01f40aaa" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:58.005519 765 raft_consensus.cc:2781] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.005576 765 raft_consensus.cc:482] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.005611 765 raft_consensus.cc:3037] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
W20240502 01:58:58.005679 29283 leader_election.cc:341] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:58:58.006065 1904 tablet_bootstrap.cc:492] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:58:58.006556 765 raft_consensus.cc:504] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.006866 765 leader_election.cc:288] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.007385 1893 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:58.007444 1904 tablet_bootstrap.cc:654] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.007459 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12420fbcddc5446b857d2e6a01f40aaa" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
W20240502 01:58:58.007809 29283 leader_election.cc:341] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:58:58.008152 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbc0190549348d29fb307282f3a7a47 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf]), partition=
I20240502 01:58:58.008352 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbc0190549348d29fb307282f3a7a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.008570 1904 tablet_bootstrap.cc:492] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:58:58.008558 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12420fbcddc5446b857d2e6a01f40aaa" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:58.008651 26927 raft_consensus.cc:3037] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.008658 1904 ts_tablet_manager.cc:1367] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:58:58.009074 1904 raft_consensus.cc:348] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.009198 1904 raft_consensus.cc:374] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.009234 1904 raft_consensus.cc:729] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.009344 1904 consensus_queue.cc:260] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.009503 26927 raft_consensus.cc:2454] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:58.009629 1904 ts_tablet_manager.cc:1398] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:58.011725 29286 leader_election.cc:302] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters: f3d18a019013487ab801599cfb80e4b4
I20240502 01:58:58.011768 1893 tablet_bootstrap.cc:654] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.012923 765 raft_consensus.cc:2781] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.013002 765 raft_consensus.cc:686] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:58.013105 765 consensus_queue.cc:237] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.016014 26510 catalog_manager.cc:5484] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:58.016872 1895 consensus_queue.cc:260] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.018121 1895 ts_tablet_manager.cc:1398] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.017s user 0.001s sys 0.000s
I20240502 01:58:58.018337 1895 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:58.019728 1895 tablet_bootstrap.cc:654] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.023339 1896 tablet_bootstrap.cc:492] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:58.023427 1896 ts_tablet_manager.cc:1367] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240502 01:58:58.023860 1896 raft_consensus.cc:348] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.023952 1896 raft_consensus.cc:374] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.023977 1896 raft_consensus.cc:729] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.024081 1896 consensus_queue.cc:260] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.024333 1896 ts_tablet_manager.cc:1398] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.024471 1896 tablet_bootstrap.cc:492] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:58.025058 1893 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:58.025135 1893 ts_tablet_manager.cc:1367] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 01:58:58.025312 1896 tablet_bootstrap.cc:654] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.025887 1893 raft_consensus.cc:348] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.025977 1893 raft_consensus.cc:374] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.026010 1893 raft_consensus.cc:729] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.026170 1893 consensus_queue.cc:260] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.027125 1893 ts_tablet_manager.cc:1398] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:58.027355 1893 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:58.028442 1893 tablet_bootstrap.cc:654] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.029724 1893 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:58.029857 1893 ts_tablet_manager.cc:1367] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:58.030368 1893 raft_consensus.cc:348] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.030570 1893 raft_consensus.cc:374] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.030668 1893 raft_consensus.cc:729] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.030827 1893 consensus_queue.cc:260] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.031224 1893 ts_tablet_manager.cc:1398] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.031419 1893 tablet_bootstrap.cc:492] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:58.032403 1893 tablet_bootstrap.cc:654] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.032430 29195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 01:58:58.033882 1896 tablet_bootstrap.cc:492] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:58.033963 1896 ts_tablet_manager.cc:1367] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 01:58:58.034416 1896 raft_consensus.cc:348] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.034526 1896 raft_consensus.cc:374] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.034564 1896 raft_consensus.cc:729] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.034690 1896 consensus_queue.cc:260] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.034951 1896 ts_tablet_manager.cc:1398] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:58.040773 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:58.044884 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:58.045183 1766 raft_consensus.cc:979] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: : Instructing follower 7c693d0933c64d9babc07f404df45cd1 to start an election
I20240502 01:58:58.045276 1766 raft_consensus.cc:1067] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Signalling peer 7c693d0933c64d9babc07f404df45cd1 to start an election
I20240502 01:58:58.048765 29487 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a326a7899c064c95b426aae61d65208f"
dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
from {username='slave'} at 127.19.43.197:52761
I20240502 01:58:58.048910 29487 raft_consensus.cc:482] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:58:58.048974 29487 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:58.049927 29487 raft_consensus.cc:504] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:58.050662 29487 leader_election.cc:288] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 2 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:58:58.050904 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:58.051000 29079 raft_consensus.cc:3032] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:58:58.051041 29079 raft_consensus.cc:729] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:58.051187 29079 consensus_queue.cc:260] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:58.051257 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:58:58.051378 26658 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:58.051383 29079 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:58.052287 29079 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 2.
I20240502 01:58:58.052371 26658 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 2.
I20240502 01:58:58.053073 29421 leader_election.cc:302] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.053246 1595 raft_consensus.cc:2781] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:58:58.053319 1595 raft_consensus.cc:686] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 2 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:58.053444 1595 consensus_queue.cc:237] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:58.054328 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:58.054440 26794 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:58:58.055356 26794 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 2.
I20240502 01:58:58.063951 26513 catalog_manager.cc:5484] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 1 to 2, leader changed from f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 2 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:58.064735 29058 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.064908 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:58.065301 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.066172 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:58.067342 29468 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:58.071249 1893 tablet_bootstrap.cc:492] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:58.071333 1893 ts_tablet_manager.cc:1367] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20240502 01:58:58.071436 1895 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:58.071513 1895 ts_tablet_manager.cc:1367] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20240502 01:58:58.071738 1893 raft_consensus.cc:348] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.071828 1893 raft_consensus.cc:374] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.071856 1893 raft_consensus.cc:729] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.071951 1893 consensus_queue.cc:260] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.071915 1895 raft_consensus.cc:348] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.072023 1895 raft_consensus.cc:374] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.072058 1895 raft_consensus.cc:729] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.072178 1893 ts_tablet_manager.cc:1398] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.072186 1895 consensus_queue.cc:260] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.073711 1895 ts_tablet_manager.cc:1398] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:58.098462 1217 raft_consensus.cc:482] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.098556 1217 raft_consensus.cc:504] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.098939 1217 leader_election.cc:288] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.102396 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbc0190549348d29fb307282f3a7a47" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:58.102452 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbc0190549348d29fb307282f3a7a47" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:58.102538 29216 raft_consensus.cc:2454] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:58.102569 29079 raft_consensus.cc:2454] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:58:58.102792 26594 leader_election.cc:302] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:58:58.102962 1217 raft_consensus.cc:2781] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.103026 1217 raft_consensus.cc:482] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.103070 1217 raft_consensus.cc:3037] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.104099 1217 raft_consensus.cc:504] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.104450 1217 leader_election.cc:288] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.104740 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbc0190549348d29fb307282f3a7a47" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:58.104840 29079 raft_consensus.cc:3037] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.104894 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbc0190549348d29fb307282f3a7a47" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:58.104995 29216 raft_consensus.cc:3037] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.105685 29079 raft_consensus.cc:2454] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:58.105705 29216 raft_consensus.cc:2454] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:58:58.105934 26592 leader_election.cc:302] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.106264 1217 raft_consensus.cc:2781] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.106364 1217 raft_consensus.cc:686] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:58:58.106492 1217 consensus_queue.cc:237] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.108507 26510 catalog_manager.cc:5484] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:58.159476 1595 raft_consensus.cc:482] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.159572 1595 raft_consensus.cc:504] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.159899 1595 leader_election.cc:288] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:58.160197 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:58.160329 29349 raft_consensus.cc:2454] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:58.160333 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:58.160470 29216 raft_consensus.cc:2454] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:58.160616 29419 leader_election.cc:302] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:58.160833 1788 raft_consensus.cc:482] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.160920 1788 raft_consensus.cc:504] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.161320 1788 leader_election.cc:288] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.162158 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499d2ca7bffc42748591db5280f829f2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:58.162190 1595 raft_consensus.cc:2781] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.162256 1595 raft_consensus.cc:482] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.162295 1595 raft_consensus.cc:3037] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.162281 29079 raft_consensus.cc:2454] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:58.162284 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499d2ca7bffc42748591db5280f829f2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:58.162381 29349 raft_consensus.cc:2454] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:58.163161 29147 leader_election.cc:302] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:58.163568 1788 raft_consensus.cc:2781] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.163630 1788 raft_consensus.cc:482] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.163666 1788 raft_consensus.cc:3037] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.164655 1788 raft_consensus.cc:504] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.164992 1788 leader_election.cc:288] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.166213 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499d2ca7bffc42748591db5280f829f2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:58.166258 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499d2ca7bffc42748591db5280f829f2" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:58.166322 29349 raft_consensus.cc:3037] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.166355 29079 raft_consensus.cc:3037] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.167063 29079 raft_consensus.cc:2454] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:58.167307 29349 raft_consensus.cc:2454] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:58.167368 29147 leader_election.cc:302] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.167527 1788 raft_consensus.cc:2781] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.167601 1788 raft_consensus.cc:686] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:58.167717 1788 consensus_queue.cc:237] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.168337 1595 raft_consensus.cc:504] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.168678 1595 leader_election.cc:288] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:58.169031 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:58.169135 29216 raft_consensus.cc:3037] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.170084 29216 raft_consensus.cc:2454] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:58.170504 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:58.170603 29349 raft_consensus.cc:3037] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.170847 29421 leader_election.cc:302] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:58.171566 29349 raft_consensus.cc:2454] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:58.172209 26513 catalog_manager.cc:5484] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:58:58.174125 1595 raft_consensus.cc:2781] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.174211 1595 raft_consensus.cc:686] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:58.174337 1595 consensus_queue.cc:237] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.176784 26508 catalog_manager.cc:5484] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:58.217049 26508 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:58.218578 29468 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:58.219106 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:58.221916 29079 raft_consensus.cc:1261] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:58.222582 29216 raft_consensus.cc:1261] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:58.223297 31766 consensus_queue.cc:1035] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.223567 31766 consensus_queue.cc:1035] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.228925 1033 mvcc.cc:205] Tried to move back new op lower bound from 7023063606149844992 to 7023063605685522432. Current Snapshot: MvccSnapshot[applied={T|T < 7023063606149844992}]
I20240502 01:58:58.228885 1724 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:58.228960 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:58.229239 1724 tablet.cc:1620] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:58.229956 26509 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 8bbc0190549348d29fb307282f3a7a47 (table rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf]). Expected version 1 got 0
I20240502 01:58:58.230911 26508 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 8bbc0190549348d29fb307282f3a7a47 (table rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf]). Expected version 1 got 0
I20240502 01:58:58.235566 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf] alter complete (version 1)
I20240502 01:58:58.237165 29058 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.238535 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:58.245721 1988 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:58.246059 1988 tablet.cc:1620] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:58:58.247112 26508 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 8bbc0190549348d29fb307282f3a7a47 (table rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf]). Expected version 1 got 0
I20240502 01:58:58.247762 1989 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:58.248452 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:58.248570 1989 tablet.cc:1620] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:58.250526 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.253366 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:58.320264 969 raft_consensus.cc:482] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.320365 969 raft_consensus.cc:504] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:58:58.320721 969 leader_election.cc:288] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:58.320986 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12420fbcddc5446b857d2e6a01f40aaa" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:58.321066 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12420fbcddc5446b857d2e6a01f40aaa" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:58.321218 26927 raft_consensus.cc:2379] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3d18a019013487ab801599cfb80e4b4 in current term 1: Already voted for candidate 468dd27456c64d67b22e8a58af93d93d in this term.
I20240502 01:58:58.321476 29016 leader_election.cc:302] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d18a019013487ab801599cfb80e4b4; no voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:58:58.321502 27064 consensus_queue.cc:237] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:58.321636 969 raft_consensus.cc:3037] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.322713 969 raft_consensus.cc:2726] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 01:58:58.322726 29349 raft_consensus.cc:1261] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:58.322806 26794 raft_consensus.cc:1261] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:58.323259 1299 consensus_queue.cc:1035] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:58.323539 1402 consensus_queue.cc:1035] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:58.324998 2008 raft_consensus.cc:2932] T 3d53d3130fa84270ad934353a5dbab41 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.325438 26794 raft_consensus.cc:2932] T 3d53d3130fa84270ad934353a5dbab41 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.325486 29349 raft_consensus.cc:2932] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.328219 26508 catalog_manager.cc:5484] T 3d53d3130fa84270ad934353a5dbab41 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.332185 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d53d3130fa84270ad934353a5dbab41 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:58.332551 2014 tablet_replica.cc:330] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:58.332698 2014 raft_consensus.cc:2229] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:58.332902 2014 raft_consensus.cc:2258] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:58.333580 2014 ts_tablet_manager.cc:1875] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:58.335922 2014 ts_tablet_manager.cc:1888] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:58.335992 2014 log.cc:1196] T 3d53d3130fa84270ad934353a5dbab41 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/3d53d3130fa84270ad934353a5dbab41
I20240502 01:58:58.336453 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 3d53d3130fa84270ad934353a5dbab41 (table rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]) successfully deleted
I20240502 01:58:58.337304 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:58.342258 1988 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:58.342566 1988 tablet.cc:1620] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:58:58.342739 1724 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:58.343032 1724 tablet.cc:1620] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:58.345069 1989 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:58.345361 1989 tablet.cc:1620] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:58.347741 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=63bbe3bdc350421dadbae6fd56346ebf] alter complete (version 2)
I20240502 01:58:58.429260 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:58.430913 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:58.432166 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:58.433885 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.441983 29058 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.444175 29468 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:58.450311 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:58.456390 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:58.456598 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:58.457959 1988 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:58.461791 1724 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:58.463268 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=63bbe3bdc350421dadbae6fd56346ebf] alter complete (version 3)
I20240502 01:58:58.464078 1989 tablet.cc:1722] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:58.464978 26793 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:58:58.465380 32697 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:58.508730 32364 consensus_queue.cc:1035] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.511372 32364 consensus_queue.cc:1035] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.519629 29080 consensus_queue.cc:237] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:58:58.520730 27064 raft_consensus.cc:1261] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:58.520929 29216 raft_consensus.cc:1261] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:58:58.521136 969 consensus_queue.cc:1035] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:58.521450 969 consensus_queue.cc:1035] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:58.522951 969 raft_consensus.cc:2932] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:58.523499 27063 raft_consensus.cc:2932] T a0ea54806f7342e2bb3d2f287bb84bdc P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:58.525511 29216 raft_consensus.cc:2932] T a0ea54806f7342e2bb3d2f287bb84bdc P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:58.526322 26509 catalog_manager.cc:5484] T a0ea54806f7342e2bb3d2f287bb84bdc P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:58:58.529006 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ea54806f7342e2bb3d2f287bb84bdc with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:58.529338 2088 tablet_replica.cc:330] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:58.529479 2088 raft_consensus.cc:2229] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:58.529594 2088 raft_consensus.cc:2258] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:58.529958 2088 ts_tablet_manager.cc:1875] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:58.532009 2088 ts_tablet_manager.cc:1888] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:58:58.532085 2088 log.cc:1196] T a0ea54806f7342e2bb3d2f287bb84bdc P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/a0ea54806f7342e2bb3d2f287bb84bdc
I20240502 01:58:58.532615 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet a0ea54806f7342e2bb3d2f287bb84bdc (table rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]) successfully deleted
I20240502 01:58:58.535104 29080 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:58:58.535522 32697 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:58.539593 26658 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:58:58.540189 32697 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:58:58.568663 26508 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:58.568802 26508 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:58.571132 26508 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 8bbc0190549348d29fb307282f3a7a47
I20240502 01:58:58.571794 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bbc0190549348d29fb307282f3a7a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:58 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:58:58.571794 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bbc0190549348d29fb307282f3a7a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:58 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:58.571966 2088 tablet_replica.cc:330] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:58:58.572084 2088 raft_consensus.cc:2229] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:58.572086 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bbc0190549348d29fb307282f3a7a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:58 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:58.572244 2088 raft_consensus.cc:2258] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:58.572748 2121 tablet_replica.cc:330] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:58.572922 2121 raft_consensus.cc:2229] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:58.572969 2088 ts_tablet_manager.cc:1875] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:58.573076 2121 raft_consensus.cc:2258] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:58.573318 2120 tablet_replica.cc:330] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:58.573421 2120 raft_consensus.cc:2229] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:58.573513 2120 raft_consensus.cc:2258] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:58.574111 2120 ts_tablet_manager.cc:1875] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:58.575172 2088 ts_tablet_manager.cc:1888] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:58.575237 2088 log.cc:1196] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/8bbc0190549348d29fb307282f3a7a47
I20240502 01:58:58.575472 2088 ts_tablet_manager.cc:1909] T 8bbc0190549348d29fb307282f3a7a47 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:58:58.576221 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 8bbc0190549348d29fb307282f3a7a47 (table rebalancer_dynamic_table_new_name [id=63bbe3bdc350421dadbae6fd56346ebf]) successfully deleted
I20240502 01:58:58.576328 2120 ts_tablet_manager.cc:1888] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:58.576424 2120 log.cc:1196] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/8bbc0190549348d29fb307282f3a7a47
I20240502 01:58:58.576725 2120 ts_tablet_manager.cc:1909] T 8bbc0190549348d29fb307282f3a7a47 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:58.577881 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 8bbc0190549348d29fb307282f3a7a47 (table rebalancer_dynamic_table_new_name [id=63bbe3bdc350421dadbae6fd56346ebf]) successfully deleted
I20240502 01:58:58.581712 2121 ts_tablet_manager.cc:1875] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:58.584177 2121 ts_tablet_manager.cc:1888] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:58.584252 2121 log.cc:1196] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/8bbc0190549348d29fb307282f3a7a47
I20240502 01:58:58.584483 2121 ts_tablet_manager.cc:1909] T 8bbc0190549348d29fb307282f3a7a47 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:58.597998 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 8bbc0190549348d29fb307282f3a7a47 (table rebalancer_dynamic_table_new_name [id=63bbe3bdc350421dadbae6fd56346ebf]) successfully deleted
I20240502 01:58:58.618276 1788 consensus_queue.cc:1035] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.622952 31391 consensus_queue.cc:1035] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.640789 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:58.642701 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:58.642983 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:58.650177 29468 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:58.653067 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.659461 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:58.659902 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:58.666664 29058 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:58.674227 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:58.678473 32697 consensus_queue.cc:1035] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.684991 26908 tablet_service.cc:1467] Processing CreateTablet for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]), partition=
I20240502 01:58:58.685005 29058 tablet_service.cc:1467] Processing CreateTablet for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]), partition=
I20240502 01:58:58.685027 29195 tablet_service.cc:1467] Processing CreateTablet for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]), partition=
I20240502 01:58:58.685232 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdf6d08bf62a4a76acd0aae24585d3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.685232 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdf6d08bf62a4a76acd0aae24585d3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.685238 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdf6d08bf62a4a76acd0aae24585d3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.687682 2159 tablet_bootstrap.cc:492] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:58.688710 2159 tablet_bootstrap.cc:654] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.689901 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:58.695358 2159 tablet_bootstrap.cc:492] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:58.695477 2159 ts_tablet_manager.cc:1367] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:58:58.696188 2159 raft_consensus.cc:348] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.696323 2159 raft_consensus.cc:374] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.696352 2159 raft_consensus.cc:729] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.696486 2159 consensus_queue.cc:260] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.696831 2159 ts_tablet_manager.cc:1398] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.698396 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 4caef235274947e48765e1c11669112a (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.698544 29468 tablet_service.cc:1467] Processing CreateTablet for tablet f518620fa0894706b44e22c7f3c51774 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.698626 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4caef235274947e48765e1c11669112a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.698787 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f518620fa0894706b44e22c7f3c51774. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.700363 2158 tablet_bootstrap.cc:492] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:58.701402 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 4caef235274947e48765e1c11669112a (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.701635 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4caef235274947e48765e1c11669112a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.702126 29058 tablet_service.cc:1467] Processing CreateTablet for tablet f518620fa0894706b44e22c7f3c51774 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.702333 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f518620fa0894706b44e22c7f3c51774. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.703667 29329 tablet_service.cc:1467] Processing CreateTablet for tablet fab0d9a2f05547e788258e5a486a7dd2 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.703917 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab0d9a2f05547e788258e5a486a7dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.703979 26908 tablet_service.cc:1467] Processing CreateTablet for tablet f518620fa0894706b44e22c7f3c51774 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.704180 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f518620fa0894706b44e22c7f3c51774. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.704291 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 4caef235274947e48765e1c11669112a (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.704483 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4caef235274947e48765e1c11669112a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.706115 2159 tablet_bootstrap.cc:492] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:58.707162 2159 tablet_bootstrap.cc:654] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.709720 1595 consensus_queue.cc:1035] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.710824 2161 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:58.712191 2161 tablet_bootstrap.cc:654] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.712738 2162 tablet_bootstrap.cc:492] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:58.713550 2162 tablet_bootstrap.cc:654] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.714183 2158 tablet_bootstrap.cc:654] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.714540 26908 tablet_service.cc:1467] Processing CreateTablet for tablet fab0d9a2f05547e788258e5a486a7dd2 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.714766 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab0d9a2f05547e788258e5a486a7dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.714965 2157 tablet_bootstrap.cc:492] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:58.715523 2159 tablet_bootstrap.cc:492] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:58.715595 2159 ts_tablet_manager.cc:1367] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240502 01:58:58.715883 2157 tablet_bootstrap.cc:654] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.715998 2159 raft_consensus.cc:348] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.716090 2159 raft_consensus.cc:374] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.716117 2159 raft_consensus.cc:729] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.716217 2159 consensus_queue.cc:260] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.716560 2159 ts_tablet_manager.cc:1398] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:58.716665 29468 tablet_service.cc:1467] Processing CreateTablet for tablet fab0d9a2f05547e788258e5a486a7dd2 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=6d883fc7f7434cc68481072549e1fb6b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:58.716761 2159 tablet_bootstrap.cc:492] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:58:58.716884 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab0d9a2f05547e788258e5a486a7dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:58.717554 2159 tablet_bootstrap.cc:654] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.718057 2158 tablet_bootstrap.cc:492] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:58.718135 2158 ts_tablet_manager.cc:1367] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 01:58:58.718968 2162 tablet_bootstrap.cc:492] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:58.719094 2162 ts_tablet_manager.cc:1367] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 01:58:58.719574 2162 raft_consensus.cc:348] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.719728 2162 raft_consensus.cc:374] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.719801 2162 raft_consensus.cc:729] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.719978 2162 consensus_queue.cc:260] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.720430 2162 ts_tablet_manager.cc:1398] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:58.721606 2159 tablet_bootstrap.cc:492] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:58:58.721679 2159 ts_tablet_manager.cc:1367] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 01:58:58.722055 2159 raft_consensus.cc:348] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.722138 2159 raft_consensus.cc:374] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.722164 2159 raft_consensus.cc:729] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.722258 2159 consensus_queue.cc:260] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.723639 2159 ts_tablet_manager.cc:1398] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:58.724588 2158 raft_consensus.cc:348] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.724746 2158 raft_consensus.cc:374] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.724833 2158 raft_consensus.cc:729] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.725013 2158 consensus_queue.cc:260] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.725236 2161 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:58.725311 2158 ts_tablet_manager.cc:1398] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:58.725327 2161 ts_tablet_manager.cc:1367] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:58:58.725466 2158 tablet_bootstrap.cc:492] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:58.725744 2161 raft_consensus.cc:348] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.725988 2161 raft_consensus.cc:374] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.726140 2161 raft_consensus.cc:729] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.726312 2161 consensus_queue.cc:260] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.727046 2157 tablet_bootstrap.cc:492] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:58.727421 2161 ts_tablet_manager.cc:1398] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:58.727659 2161 tablet_bootstrap.cc:492] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:58.728096 2158 tablet_bootstrap.cc:654] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.728526 2161 tablet_bootstrap.cc:654] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.728667 2157 ts_tablet_manager.cc:1367] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:58.729135 2157 raft_consensus.cc:348] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.729290 2157 raft_consensus.cc:374] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.729501 2157 raft_consensus.cc:729] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.729667 2157 consensus_queue.cc:260] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.730051 2157 ts_tablet_manager.cc:1398] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.730268 2157 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:58.730317 2161 tablet_bootstrap.cc:492] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:58.730446 2161 ts_tablet_manager.cc:1367] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:58.731166 2157 tablet_bootstrap.cc:654] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.731626 2161 raft_consensus.cc:348] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.731842 2161 raft_consensus.cc:374] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.731925 2161 raft_consensus.cc:729] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.732406 2161 consensus_queue.cc:260] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.732721 2161 ts_tablet_manager.cc:1398] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:58.732924 2161 tablet_bootstrap.cc:492] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:58.733074 2157 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:58.733091 2158 tablet_bootstrap.cc:492] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:58.733161 2158 ts_tablet_manager.cc:1367] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:58:58.733191 2157 ts_tablet_manager.cc:1367] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:58:58.733714 2158 raft_consensus.cc:348] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.733871 2158 raft_consensus.cc:374] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.733943 2158 raft_consensus.cc:729] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.734122 2158 consensus_queue.cc:260] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.734515 2158 ts_tablet_manager.cc:1398] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:58.734939 2157 raft_consensus.cc:348] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.735498 2157 raft_consensus.cc:374] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.735574 2157 raft_consensus.cc:729] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.736603 2157 consensus_queue.cc:260] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.737021 2157 ts_tablet_manager.cc:1398] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:58:58.738446 2158 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:58.739459 2158 tablet_bootstrap.cc:654] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.739990 2161 tablet_bootstrap.cc:654] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:58.740587 2158 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:58.740703 2158 ts_tablet_manager.cc:1367] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:58.741562 1595 raft_consensus.cc:482] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.741642 1595 raft_consensus.cc:504] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.741701 2158 raft_consensus.cc:348] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.741804 2158 raft_consensus.cc:374] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.741835 2158 raft_consensus.cc:729] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.741963 1595 leader_election.cc:288] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:58.741961 2158 consensus_queue.cc:260] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.742455 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:58.742566 26927 raft_consensus.cc:2454] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:58.742748 2158 ts_tablet_manager.cc:1398] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.742904 29422 leader_election.cc:302] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:58.743113 1595 raft_consensus.cc:2781] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.743160 1595 raft_consensus.cc:482] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.743191 1595 raft_consensus.cc:3037] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.743605 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:58.743918 29080 raft_consensus.cc:2454] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:58.744069 1595 raft_consensus.cc:504] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.744388 1595 leader_election.cc:288] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:58.744654 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:58.744750 29080 raft_consensus.cc:3037] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.744813 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:58.744895 26927 raft_consensus.cc:3037] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.745618 29080 raft_consensus.cc:2454] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:58.745682 26927 raft_consensus.cc:2454] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:58.745918 29422 leader_election.cc:302] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:58.746136 1595 raft_consensus.cc:2781] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.746227 1595 raft_consensus.cc:686] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:58.746362 1595 consensus_queue.cc:237] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.747296 2161 tablet_bootstrap.cc:492] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:58.747383 2161 ts_tablet_manager.cc:1367] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240502 01:58:58.747778 2161 raft_consensus.cc:348] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.747872 2161 raft_consensus.cc:374] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:58.747916 2161 raft_consensus.cc:729] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:58.748016 2161 consensus_queue.cc:260] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.748255 2161 ts_tablet_manager.cc:1398] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:58.748378 26511 catalog_manager.cc:5484] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:58.756862 27063 consensus_queue.cc:237] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:58.758096 29349 raft_consensus.cc:1261] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:58.758302 26661 raft_consensus.cc:1261] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:58.758502 1402 consensus_queue.cc:1035] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:58.758669 2008 consensus_queue.cc:1035] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:58.760486 2008 raft_consensus.cc:2932] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.760793 26661 raft_consensus.cc:2932] T 47f84b6ffb074bc682aa7b087b8c3614 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.763072 26511 catalog_manager.cc:5484] T 47f84b6ffb074bc682aa7b087b8c3614 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.764379 29350 raft_consensus.cc:2932] T 47f84b6ffb074bc682aa7b087b8c3614 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:58.767745 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f84b6ffb074bc682aa7b087b8c3614 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:58.768031 2014 tablet_replica.cc:330] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:58.768165 2014 raft_consensus.cc:2229] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:58.768308 2014 raft_consensus.cc:2258] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:58.768857 2014 ts_tablet_manager.cc:1875] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:58.770556 2014 ts_tablet_manager.cc:1888] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:58.770629 2014 log.cc:1196] T 47f84b6ffb074bc682aa7b087b8c3614 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/47f84b6ffb074bc682aa7b087b8c3614
I20240502 01:58:58.771049 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 47f84b6ffb074bc682aa7b087b8c3614 (table rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]) successfully deleted
I20240502 01:58:58.825708 1788 raft_consensus.cc:482] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.825824 1788 raft_consensus.cc:504] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.826246 1788 leader_election.cc:288] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.826686 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdf6d08bf62a4a76acd0aae24585d3a6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:58.826834 26930 raft_consensus.cc:2454] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:58.827051 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdf6d08bf62a4a76acd0aae24585d3a6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:58:58.827203 29080 raft_consensus.cc:2454] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:58.827642 29147 leader_election.cc:302] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.827817 1788 raft_consensus.cc:2781] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.827886 1788 raft_consensus.cc:482] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.827926 1788 raft_consensus.cc:3037] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.828991 1788 raft_consensus.cc:504] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.829393 1788 leader_election.cc:288] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:58:58.829722 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdf6d08bf62a4a76acd0aae24585d3a6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:58.829831 26930 raft_consensus.cc:3037] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.830840 26930 raft_consensus.cc:2454] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:58.831708 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdf6d08bf62a4a76acd0aae24585d3a6" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:58:58.831806 29080 raft_consensus.cc:3037] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.832708 29080 raft_consensus.cc:2454] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:58.832991 29147 leader_election.cc:302] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.833184 1788 raft_consensus.cc:2781] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.833258 1788 raft_consensus.cc:686] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:58.833376 1788 consensus_queue.cc:237] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.836601 26506 catalog_manager.cc:5484] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:58.871989 969 raft_consensus.cc:482] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.872079 969 raft_consensus.cc:504] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.872417 969 leader_election.cc:288] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:58.872730 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4caef235274947e48765e1c11669112a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:58.872759 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4caef235274947e48765e1c11669112a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:58.872869 29489 raft_consensus.cc:2454] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:58.872869 29216 raft_consensus.cc:2454] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:58.873170 29016 leader_election.cc:302] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.873339 969 raft_consensus.cc:2781] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.873407 969 raft_consensus.cc:482] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.873435 969 raft_consensus.cc:3037] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.874466 969 raft_consensus.cc:504] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.874773 969 leader_election.cc:288] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:58.875053 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4caef235274947e48765e1c11669112a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:58.875065 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4caef235274947e48765e1c11669112a" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:58.875151 29216 raft_consensus.cc:3037] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.875172 29489 raft_consensus.cc:3037] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.875849 29489 raft_consensus.cc:2454] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:58.875869 29216 raft_consensus.cc:2454] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:58.876125 29015 leader_election.cc:302] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:58.876291 969 raft_consensus.cc:2781] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.876359 969 raft_consensus.cc:686] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:58.876475 969 consensus_queue.cc:237] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:58.878609 26510 catalog_manager.cc:5484] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:58.966308 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:58.970099 29080 raft_consensus.cc:1261] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:58.970165 26930 raft_consensus.cc:1261] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:58.970521 31391 consensus_queue.cc:1035] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.970731 1788 consensus_queue.cc:1035] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:58.973532 26510 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]). Expected version 1 got 0
I20240502 01:58:58.974589 26510 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]). Expected version 1 got 0
I20240502 01:58:58.976938 2254 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:58.977059 1595 raft_consensus.cc:482] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:58.977139 1595 raft_consensus.cc:504] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.977239 2254 tablet.cc:1620] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:58.977480 1595 leader_election.cc:288] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:58.977716 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:58.977854 29349 raft_consensus.cc:2454] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:58.978138 29419 leader_election.cc:302] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:58.978317 1595 raft_consensus.cc:2781] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:58.978367 1595 raft_consensus.cc:482] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:58.978401 1595 raft_consensus.cc:3037] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.978514 26509 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]). Expected version 1 got 0
I20240502 01:58:58.979166 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:58:58.979283 26930 raft_consensus.cc:2454] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:58:58.979365 1595 raft_consensus.cc:504] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.979710 1595 leader_election.cc:288] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:58.980126 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:58.980178 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:58:58.980226 29349 raft_consensus.cc:3037] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.980271 26930 raft_consensus.cc:3037] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:58.981148 29349 raft_consensus.cc:2454] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:58.981159 26930 raft_consensus.cc:2454] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:58:58.981596 29422 leader_election.cc:302] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:58:58.982368 1595 raft_consensus.cc:2781] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:58.982447 1595 raft_consensus.cc:686] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:58:58.982579 1595 consensus_queue.cc:237] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:58:58.983424 2253 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:58.983750 2253 tablet.cc:1620] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:58.984591 26510 catalog_manager.cc:5484] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:58:58.984889 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684] alter complete (version 1)
I20240502 01:58:58.985857 26509 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684]). Expected version 1 got 0
I20240502 01:58:58.986176 2255 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:58.987493 2255 tablet.cc:1620] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:59.004884 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:59.006119 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:59.006121 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.009919 29195 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:59.011612 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:59.022862 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:59.026351 29468 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.027933 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:59.083882 26930 consensus_queue.cc:237] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:59.085310 29489 raft_consensus.cc:1261] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:59.085384 27063 raft_consensus.cc:1261] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:59.085685 2063 consensus_queue.cc:1035] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:59.086340 2063 consensus_queue.cc:1035] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:59.087378 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:58:59.087656 2268 raft_consensus.cc:2932] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.088312 29489 raft_consensus.cc:2932] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.090202 27063 raft_consensus.cc:2932] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.090651 26509 catalog_manager.cc:5484] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.092957 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:59.094197 2288 tablet_replica.cc:330] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:59.094408 2288 raft_consensus.cc:2229] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:59.095366 2253 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:59.095690 2253 tablet.cc:1620] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:59.095813 2254 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:59.096109 2254 tablet.cc:1620] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:58:59.098934 2288 raft_consensus.cc:2258] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:59.099710 2288 ts_tablet_manager.cc:1875] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:59.099835 2255 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:58:59.100114 2255 tablet.cc:1620] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:58:59.101644 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=bbff124fdedc46b6827a69226e066684] alter complete (version 2)
I20240502 01:58:59.102496 2288 ts_tablet_manager.cc:1888] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:59.102701 2288 log.cc:1196] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/4a528d9bbfa2488e8ce1e7b99b9e4d49
I20240502 01:58:59.103282 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49 (table rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]) successfully deleted
I20240502 01:58:59.122462 26930 consensus_queue.cc:237] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:59.123757 29216 raft_consensus.cc:1261] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:59.123804 29349 raft_consensus.cc:1261] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:59.124186 2310 consensus_queue.cc:1035] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:58:59.124476 2268 consensus_queue.cc:1035] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:58:59.124636 26861 consensus_peers.cc:485] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d14b1389d460401b83d3652b25aaddec. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:58:59.126420 2268 raft_consensus.cc:2932] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:59.126859 29349 raft_consensus.cc:2932] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:59.128875 26510 catalog_manager.cc:5484] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:59.129393 29216 raft_consensus.cc:2932] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:58:59.204674 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:58:59.210959 2253 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:59.215499 2255 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:59.216259 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=bbff124fdedc46b6827a69226e066684] alter complete (version 3)
I20240502 01:58:59.219197 2254 tablet.cc:1722] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:58:59.236048 29195 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:59.236742 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:59.243217 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:59.250214 29468 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.251173 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.263306 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:59.266779 1595 consensus_queue.cc:1035] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.270079 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:59.270190 2101 consensus_queue.cc:1035] T f518620fa0894706b44e22c7f3c51774 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.289512 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:59.320818 26511 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:59.320955 26511 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:58:59.323182 26511 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet fdf6d08bf62a4a76acd0aae24585d3a6
I20240502 01:58:59.323768 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet fdf6d08bf62a4a76acd0aae24585d3a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:59 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:58:59.324107 2374 tablet_replica.cc:330] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:58:59.324086 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet fdf6d08bf62a4a76acd0aae24585d3a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:59 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:58:59.324282 2374 raft_consensus.cc:2229] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:58:59.324499 2374 raft_consensus.cc:2258] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:59.325057 2374 ts_tablet_manager.cc:1875] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:59.325286 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet fdf6d08bf62a4a76acd0aae24585d3a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:58:59 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:59.325420 2375 tablet_replica.cc:330] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:58:59.325543 2375 raft_consensus.cc:2229] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:59.325773 2375 raft_consensus.cc:2258] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:59.326092 2376 tablet_replica.cc:330] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:59.326264 2376 raft_consensus.cc:2229] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:59.326432 2376 raft_consensus.cc:2258] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:59.326771 2375 ts_tablet_manager.cc:1875] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:59.327064 2376 ts_tablet_manager.cc:1875] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:58:59.327950 2374 ts_tablet_manager.cc:1888] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:59.328061 2374 log.cc:1196] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/fdf6d08bf62a4a76acd0aae24585d3a6
I20240502 01:58:59.328356 2374 ts_tablet_manager.cc:1909] T fdf6d08bf62a4a76acd0aae24585d3a6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:58:59.329115 2376 ts_tablet_manager.cc:1888] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:59.329110 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table_new_name [id=bbff124fdedc46b6827a69226e066684]) successfully deleted
I20240502 01:58:59.329195 2376 log.cc:1196] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/fdf6d08bf62a4a76acd0aae24585d3a6
I20240502 01:58:59.329497 2376 ts_tablet_manager.cc:1909] T fdf6d08bf62a4a76acd0aae24585d3a6 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:58:59.330106 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table_new_name [id=bbff124fdedc46b6827a69226e066684]) successfully deleted
I20240502 01:58:59.330655 2375 ts_tablet_manager.cc:1888] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:58:59.330731 2375 log.cc:1196] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/fdf6d08bf62a4a76acd0aae24585d3a6
I20240502 01:58:59.330937 2375 ts_tablet_manager.cc:1909] T fdf6d08bf62a4a76acd0aae24585d3a6 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:58:59.331485 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet fdf6d08bf62a4a76acd0aae24585d3a6 (table rebalancer_dynamic_table_new_name [id=bbff124fdedc46b6827a69226e066684]) successfully deleted
I20240502 01:58:59.413822 29195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 01:58:59.421490 29329 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:58:59.424882 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:58:59.427510 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:59.430963 29468 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.434840 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:59.438647 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 497bba35b4e341a193b34c617a59e183 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]), partition=
I20240502 01:58:59.438902 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497bba35b4e341a193b34c617a59e183. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.440294 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 497bba35b4e341a193b34c617a59e183 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]), partition=
I20240502 01:58:59.440515 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497bba35b4e341a193b34c617a59e183. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.441280 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 497bba35b4e341a193b34c617a59e183 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]), partition=
I20240502 01:58:59.441498 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497bba35b4e341a193b34c617a59e183. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.443243 2426 tablet_bootstrap.cc:492] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:59.444051 2426 tablet_bootstrap.cc:654] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.445288 2087 consensus_queue.cc:1035] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.446249 29059 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:58:59.456771 2426 tablet_bootstrap.cc:492] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:58:59.456877 2426 ts_tablet_manager.cc:1367] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:58:59.459160 2426 raft_consensus.cc:348] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.459375 2426 raft_consensus.cc:374] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.459441 2426 raft_consensus.cc:729] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.459504 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:59.459558 2426 consensus_queue.cc:260] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.460023 2426 ts_tablet_manager.cc:1398] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:59.462599 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.467072 2427 tablet_bootstrap.cc:492] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:59.468052 2427 tablet_bootstrap.cc:654] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.471132 2087 consensus_queue.cc:1035] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.481241 2101 consensus_queue.cc:1035] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.485751 2425 tablet_bootstrap.cc:492] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:59.486733 2425 tablet_bootstrap.cc:654] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.487207 2101 consensus_queue.cc:1035] T fab0d9a2f05547e788258e5a486a7dd2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.490525 2425 tablet_bootstrap.cc:492] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:59.490667 2425 ts_tablet_manager.cc:1367] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:58:59.491143 2425 raft_consensus.cc:348] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.491310 2425 raft_consensus.cc:374] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.491389 2425 raft_consensus.cc:729] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.491578 2425 consensus_queue.cc:260] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.492076 2425 ts_tablet_manager.cc:1398] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:59.493317 2427 tablet_bootstrap.cc:492] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:59.493407 2427 ts_tablet_manager.cc:1367] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 01:58:59.493852 2427 raft_consensus.cc:348] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.493959 2427 raft_consensus.cc:374] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.493988 2427 raft_consensus.cc:729] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.494115 2427 consensus_queue.cc:260] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.494374 2427 ts_tablet_manager.cc:1398] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:59.513911 26927 consensus_queue.cc:237] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:58:59.515079 29489 raft_consensus.cc:1261] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:59.515110 27063 raft_consensus.cc:1261] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:59.515530 2313 consensus_queue.cc:1035] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:59.515733 2063 consensus_queue.cc:1035] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:59.516304 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:58:59.517469 2310 raft_consensus.cc:2932] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.517988 27063 raft_consensus.cc:2932] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.518920 29489 raft_consensus.cc:2932] T 2c150446ded240a2946d5afa10ecd1df P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.523038 26511 catalog_manager.cc:5484] T 2c150446ded240a2946d5afa10ecd1df P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 5, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:58:59.526388 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.526683 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.527072 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 11a922c5a8994e92a4112d89ce8a9f3a (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.527345 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a922c5a8994e92a4112d89ce8a9f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.527400 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 11a922c5a8994e92a4112d89ce8a9f3a (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.527638 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a922c5a8994e92a4112d89ce8a9f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.528918 2456 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:58:59.528930 2427 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:59.529055 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.529260 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.529665 2425 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:59.529793 2427 tablet_bootstrap.cc:654] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.529808 2456 tablet_bootstrap.cc:654] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.530498 2425 tablet_bootstrap.cc:654] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.531252 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.531489 2425 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:59.531474 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.531566 2425 ts_tablet_manager.cc:1367] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:59.531929 2425 raft_consensus.cc:348] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.532019 2425 raft_consensus.cc:374] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.532052 2425 raft_consensus.cc:729] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.532148 2425 consensus_queue.cc:260] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.532214 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c150446ded240a2946d5afa10ecd1df with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44132
I20240502 01:58:59.532419 2425 ts_tablet_manager.cc:1398] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:59.532567 2288 tablet_replica.cc:330] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:58:59.532697 2288 raft_consensus.cc:2229] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:59.532858 2288 raft_consensus.cc:2258] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:59.532867 29329 tablet_service.cc:1467] Processing CreateTablet for tablet c52dc450e0ca4a68af65e924d0e99ce3 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.533463 2288 ts_tablet_manager.cc:1875] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:59.535207 2427 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:59.535290 2427 ts_tablet_manager.cc:1367] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 01:58:59.535601 2288 ts_tablet_manager.cc:1888] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:59.535660 2288 log.cc:1196] T 2c150446ded240a2946d5afa10ecd1df P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/2c150446ded240a2946d5afa10ecd1df
I20240502 01:58:59.535702 2427 raft_consensus.cc:348] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.535809 2427 raft_consensus.cc:374] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.535842 2427 raft_consensus.cc:729] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.535953 2427 consensus_queue.cc:260] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.536218 2427 ts_tablet_manager.cc:1398] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:58:59.536635 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 11a922c5a8994e92a4112d89ce8a9f3a (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.536880 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a922c5a8994e92a4112d89ce8a9f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.536875 29196 tablet_service.cc:1467] Processing CreateTablet for tablet c52dc450e0ca4a68af65e924d0e99ce3 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.537110 2425 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:59.538072 2425 tablet_bootstrap.cc:654] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.538735 2465 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:59.538829 29454 tablet_service.cc:1467] Processing CreateTablet for tablet c52dc450e0ca4a68af65e924d0e99ce3 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=cb4c07c573ee44f4bba16eb6623444ed]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:58:59.538908 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52dc450e0ca4a68af65e924d0e99ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.539018 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52dc450e0ca4a68af65e924d0e99ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.539674 2465 tablet_bootstrap.cc:654] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.540760 2465 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:59.540851 2465 ts_tablet_manager.cc:1367] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:58:59.541023 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 2c150446ded240a2946d5afa10ecd1df (table rebalance_test_table_3 [id=60a3751a3295452a8c5c748f9b0c81af]) successfully deleted
I20240502 01:58:59.541147 2425 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:59.541222 2425 ts_tablet_manager.cc:1367] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 01:58:59.541246 2465 raft_consensus.cc:348] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.541344 2465 raft_consensus.cc:374] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.541376 2465 raft_consensus.cc:729] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.541481 2465 consensus_queue.cc:260] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.541738 2465 ts_tablet_manager.cc:1398] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:59.541903 2465 tablet_bootstrap.cc:492] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:58:59.542147 2425 raft_consensus.cc:348] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.542249 2425 raft_consensus.cc:374] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.542279 2425 raft_consensus.cc:729] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.542598 2425 consensus_queue.cc:260] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.542757 2465 tablet_bootstrap.cc:654] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.542930 2425 ts_tablet_manager.cc:1398] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:59.543071 2425 tablet_bootstrap.cc:492] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:58:59.544065 2427 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:59.544230 2425 tablet_bootstrap.cc:654] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.544893 2427 tablet_bootstrap.cc:654] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.545186 2425 tablet_bootstrap.cc:492] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:58:59.545269 2425 ts_tablet_manager.cc:1367] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:58:59.545765 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52dc450e0ca4a68af65e924d0e99ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.546167 2427 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:59.546236 2427 ts_tablet_manager.cc:1367] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:58:59.546710 2427 raft_consensus.cc:348] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.547561 2427 raft_consensus.cc:374] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.547652 2427 raft_consensus.cc:729] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.547844 2427 consensus_queue.cc:260] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.548238 2427 ts_tablet_manager.cc:1398] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:58:59.548470 2465 tablet_bootstrap.cc:492] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:58:59.548563 2465 ts_tablet_manager.cc:1367] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:58:59.549492 2465 raft_consensus.cc:348] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.549594 2465 raft_consensus.cc:374] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.549630 2465 raft_consensus.cc:729] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.549741 2465 consensus_queue.cc:260] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.549225 2427 tablet_bootstrap.cc:492] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:58:59.550801 2427 tablet_bootstrap.cc:654] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:58:59.551271 2465 ts_tablet_manager.cc:1398] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:58:59.545611 2425 raft_consensus.cc:348] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.551532 2425 raft_consensus.cc:374] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.551614 2425 raft_consensus.cc:729] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.551793 2425 consensus_queue.cc:260] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.552053 2425 ts_tablet_manager.cc:1398] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:58:59.552137 2456 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:58:59.552222 2456 ts_tablet_manager.cc:1367] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240502 01:58:59.552611 2456 raft_consensus.cc:348] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.552706 2456 raft_consensus.cc:374] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.552731 2456 raft_consensus.cc:729] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.552840 2456 consensus_queue.cc:260] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.553076 2456 ts_tablet_manager.cc:1398] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:58:59.553158 2427 tablet_bootstrap.cc:492] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:58:59.553247 2427 ts_tablet_manager.cc:1367] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 01:58:59.553622 2427 raft_consensus.cc:348] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.553694 2427 raft_consensus.cc:374] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:58:59.553712 2427 raft_consensus.cc:729] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.553799 2427 consensus_queue.cc:260] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.553988 2427 ts_tablet_manager.cc:1398] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:58:59.621729 2472 raft_consensus.cc:482] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:59.621853 2472 raft_consensus.cc:504] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.622323 2472 leader_election.cc:288] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:59.622509 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52dc450e0ca4a68af65e924d0e99ce3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:59.622627 29213 raft_consensus.cc:2454] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:59.622920 29285 leader_election.cc:302] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:59.623090 2472 raft_consensus.cc:2781] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:59.623140 2472 raft_consensus.cc:482] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:59.623174 2472 raft_consensus.cc:3037] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.623665 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52dc450e0ca4a68af65e924d0e99ce3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:59.623788 29489 raft_consensus.cc:2454] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:58:59.624389 2472 raft_consensus.cc:504] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.624742 2472 leader_election.cc:288] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:58:59.625296 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52dc450e0ca4a68af65e924d0e99ce3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:59.625408 29489 raft_consensus.cc:3037] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.626385 29489 raft_consensus.cc:2454] T c52dc450e0ca4a68af65e924d0e99ce3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:59.626631 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52dc450e0ca4a68af65e924d0e99ce3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:59.626754 29213 raft_consensus.cc:3037] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.626755 29286 leader_election.cc:302] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:59.626974 2472 raft_consensus.cc:2781] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:59.627050 2472 raft_consensus.cc:686] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:58:59.627199 2472 consensus_queue.cc:237] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.627722 29213 raft_consensus.cc:2454] T c52dc450e0ca4a68af65e924d0e99ce3 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:58:59.629549 26510 catalog_manager.cc:5484] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.642891 2432 raft_consensus.cc:482] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:59.642973 2432 raft_consensus.cc:504] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.643280 2432 leader_election.cc:288] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:59.643594 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b4f8eed739249cd9fc125d5ef5a6ed8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:58:59.643620 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b4f8eed739249cd9fc125d5ef5a6ed8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:59.643692 29213 raft_consensus.cc:2454] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:59.643747 29349 raft_consensus.cc:2454] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:58:59.643924 29015 leader_election.cc:302] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:59.644116 2432 raft_consensus.cc:2781] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:59.644165 2432 raft_consensus.cc:482] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:59.644189 2432 raft_consensus.cc:3037] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.644954 2432 raft_consensus.cc:504] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.645285 2432 leader_election.cc:288] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:58:59.645574 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b4f8eed739249cd9fc125d5ef5a6ed8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:58:59.645596 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b4f8eed739249cd9fc125d5ef5a6ed8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:59.645681 29349 raft_consensus.cc:3037] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.645681 29213 raft_consensus.cc:3037] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.646610 29349 raft_consensus.cc:2454] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:59.646639 29213 raft_consensus.cc:2454] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:58:59.646914 29013 leader_election.cc:302] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:58:59.647162 2087 raft_consensus.cc:2781] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:59.647234 2087 raft_consensus.cc:686] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:58:59.647348 2087 consensus_queue.cc:237] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:58:59.649066 26513 catalog_manager.cc:5484] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:58:59.662768 2511 ts_tablet_manager.cc:902] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:58:59.663254 2511 tablet_copy_client.cc:308] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:58:59.663684 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet d14b1389d460401b83d3652b25aaddec from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:47393)
I20240502 01:58:59.663779 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet d14b1389d460401b83d3652b25aaddec from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:47393: session id = b290c1bf893444078ed721f25995d124-d14b1389d460401b83d3652b25aaddec
I20240502 01:58:59.664745 26950 tablet_copy_source_session.cc:205] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:58:59.665577 2511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14b1389d460401b83d3652b25aaddec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:58:59.667484 2511 tablet_copy_client.cc:783] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:58:59.667658 2511 tablet_copy_client.cc:647] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:58:59.669009 2511 tablet_copy_client.cc:523] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:58:59.669948 2511 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:58:59.673112 2511 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:58:59.673305 2511 tablet_bootstrap.cc:492] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:58:59.673421 2511 ts_tablet_manager.cc:1367] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:58:59.673748 2511 raft_consensus.cc:348] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:59.673854 2511 raft_consensus.cc:729] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:58:59.673974 2511 consensus_queue.cc:260] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:58:59.674280 2511 ts_tablet_manager.cc:1398] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:58:59.674934 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session b290c1bf893444078ed721f25995d124-d14b1389d460401b83d3652b25aaddec received from {username='slave'} at 127.19.43.194:47393
I20240502 01:58:59.675011 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session b290c1bf893444078ed721f25995d124-d14b1389d460401b83d3652b25aaddec on tablet d14b1389d460401b83d3652b25aaddec with peer b290c1bf893444078ed721f25995d124
I20240502 01:58:59.692559 1788 raft_consensus.cc:482] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:59.692643 1788 raft_consensus.cc:504] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.693027 1788 leader_election.cc:288] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:59.693261 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a922c5a8994e92a4112d89ce8a9f3a" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:58:59.693361 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a922c5a8994e92a4112d89ce8a9f3a" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:59.693399 29489 raft_consensus.cc:2454] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:59.693485 29349 raft_consensus.cc:2454] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:59.693740 29150 leader_election.cc:302] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:58:59.693912 1788 raft_consensus.cc:2781] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:59.693966 1788 raft_consensus.cc:482] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:59.694000 1788 raft_consensus.cc:3037] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.694685 1788 raft_consensus.cc:504] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.695016 1788 leader_election.cc:288] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:59.695267 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a922c5a8994e92a4112d89ce8a9f3a" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:59.695276 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a922c5a8994e92a4112d89ce8a9f3a" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:58:59.695371 29489 raft_consensus.cc:3037] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.695379 29349 raft_consensus.cc:3037] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.696143 29349 raft_consensus.cc:2454] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:59.696233 29489 raft_consensus.cc:2454] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:59.696573 29147 leader_election.cc:302] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:59.696727 1788 raft_consensus.cc:2781] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:59.696810 1788 raft_consensus.cc:686] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:59.696935 1788 consensus_queue.cc:237] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.698740 26508 catalog_manager.cc:5484] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.804473 1788 raft_consensus.cc:482] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:58:59.804589 1788 raft_consensus.cc:504] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.806139 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497bba35b4e341a193b34c617a59e183" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:58:59.806234 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497bba35b4e341a193b34c617a59e183" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:58:59.806274 29349 raft_consensus.cc:2454] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:59.806365 26793 raft_consensus.cc:2454] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:58:59.806499 1788 leader_election.cc:288] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:59.806628 29147 leader_election.cc:302] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:58:59.806793 1788 raft_consensus.cc:2781] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:58:59.806840 1788 raft_consensus.cc:482] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:58:59.806867 1788 raft_consensus.cc:3037] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.807732 1788 raft_consensus.cc:504] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.808045 1788 leader_election.cc:288] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:58:59.808308 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497bba35b4e341a193b34c617a59e183" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:58:59.808414 26793 raft_consensus.cc:3037] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.808411 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497bba35b4e341a193b34c617a59e183" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:58:59.808529 29349 raft_consensus.cc:3037] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:58:59.809414 26793 raft_consensus.cc:2454] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:59.809427 29349 raft_consensus.cc:2454] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:58:59.809684 29149 leader_election.cc:302] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:58:59.810460 1788 raft_consensus.cc:2781] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:58:59.810647 1788 raft_consensus.cc:686] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:58:59.810778 1788 consensus_queue.cc:237] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.812616 26513 catalog_manager.cc:5484] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.841938 29329 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:59.848487 29196 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:59.852334 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:58:59.853230 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.854609 29454 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:58:59.857792 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:58:59.878141 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:59.897439 29059 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:58:59.924073 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:58:59.928166 29349 raft_consensus.cc:1261] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:59.928274 26793 raft_consensus.cc:1261] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:58:59.929958 31391 consensus_queue.cc:1035] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.931753 31391 consensus_queue.cc:1035] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:58:59.932554 26513 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet 497bba35b4e341a193b34c617a59e183 (table rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]). Expected version 1 got 0
I20240502 01:58:59.934764 2544 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:59.935791 2544 tablet.cc:1620] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:58:59.936681 26508 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet 497bba35b4e341a193b34c617a59e183 (table rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]). Expected version 1 got 0
I20240502 01:58:59.937649 2550 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:59.937741 26513 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 497bba35b4e341a193b34c617a59e183 (table rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]). Expected version 1 got 0
I20240502 01:58:59.937934 2550 tablet.cc:1620] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:58:59.938921 2545 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:58:59.939240 2545 tablet.cc:1620] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:58:59.941613 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a] alter complete (version 1)
I20240502 01:58:59.944991 27063 consensus_queue.cc:237] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:58:59.946208 29349 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:59.946211 26661 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:58:59.946599 1402 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:59.946908 1402 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:58:59.948477 1402 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.948992 26661 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.949106 29349 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.951624 26511 catalog_manager.cc:5484] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:58:59.954071 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d96b487a964b90b720d8977a96fc5b with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:58:59.954386 2561 tablet_replica.cc:330] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:58:59.954592 2561 raft_consensus.cc:2229] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:58:59.954823 2561 raft_consensus.cc:2258] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:58:59.955461 2561 ts_tablet_manager.cc:1875] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:58:59.957445 2561 ts_tablet_manager.cc:1888] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:58:59.957523 2561 log.cc:1196] T a0d96b487a964b90b720d8977a96fc5b P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/a0d96b487a964b90b720d8977a96fc5b
I20240502 01:58:59.958068 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet a0d96b487a964b90b720d8977a96fc5b (table rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]) successfully deleted
I20240502 01:59:00.033355 2087 consensus_queue.cc:1035] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.037461 2432 consensus_queue.cc:1035] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.044533 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:00.052351 2545 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:00.052652 2545 tablet.cc:1620] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:00.052695 2544 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:00.052966 2544 tablet.cc:1620] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:00.056409 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.058997 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=3b5f990c70ae416cb1f13f3f1a9cfe1a] alter complete (version 2)
I20240502 01:59:00.059571 29329 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:00.060667 29454 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:00.061520 2550 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:00.061806 2550 tablet.cc:1620] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:00.065662 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:00.069087 29196 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:00.073995 26775 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:00.085847 29059 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:00.086270 26640 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:00.088632 1788 consensus_queue.cc:1035] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.120268 31391 consensus_queue.cc:1035] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.140201 26793 raft_consensus.cc:1203] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:00.149104 26927 consensus_queue.cc:237] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:00.150442 26793 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:00.150729 29213 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:00.151135 2313 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:00.151324 2063 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:00.153048 2313 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:00.156117 26509 catalog_manager.cc:5484] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:00.156602 26793 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:00.158820 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 626774246b9445bc838427358e08f2cd with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:00.159399 2635 tablet_replica.cc:330] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:00.159579 2635 raft_consensus.cc:2229] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:00.160454 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:00.160583 29213 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:00.159886 2635 raft_consensus.cc:2258] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.161258 2635 ts_tablet_manager.cc:1875] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:00.163810 2635 ts_tablet_manager.cc:1888] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:59:00.163904 2635 log.cc:1196] T 626774246b9445bc838427358e08f2cd P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/626774246b9445bc838427358e08f2cd
I20240502 01:59:00.165648 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 626774246b9445bc838427358e08f2cd (table rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]) successfully deleted
I20240502 01:59:00.166574 2472 consensus_queue.cc:1035] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.167610 2544 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:00.170159 2545 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:00.170912 32364 consensus_queue.cc:1035] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.172374 2550 tablet.cc:1722] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:00.174592 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=3b5f990c70ae416cb1f13f3f1a9cfe1a] alter complete (version 3)
I20240502 01:59:00.202911 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:00.208560 27044 tablet_service.cc:1467] Processing CreateTablet for tablet c528788d4f8a43a2a6614e7e9ef79fbd (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.208840 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c528788d4f8a43a2a6614e7e9ef79fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.209051 26640 tablet_service.cc:1467] Processing CreateTablet for tablet c528788d4f8a43a2a6614e7e9ef79fbd (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.209296 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c528788d4f8a43a2a6614e7e9ef79fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.209362 26775 tablet_service.cc:1467] Processing CreateTablet for tablet c528788d4f8a43a2a6614e7e9ef79fbd (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.209570 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c528788d4f8a43a2a6614e7e9ef79fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.210343 29058 tablet_service.cc:1467] Processing CreateTablet for tablet d6e70ac8b41e4127881c5ce5167866de (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.210597 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e70ac8b41e4127881c5ce5167866de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.211733 26774 tablet_service.cc:1467] Processing CreateTablet for tablet d3d8471cf7654dbdab51d855ed30be18 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.211956 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d8471cf7654dbdab51d855ed30be18. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.211993 2668 tablet_bootstrap.cc:492] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:00.212917 2668 tablet_bootstrap.cc:654] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.212929 29059 tablet_service.cc:1467] Processing CreateTablet for tablet d3d8471cf7654dbdab51d855ed30be18 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.213137 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d8471cf7654dbdab51d855ed30be18. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.214974 29454 tablet_service.cc:1467] Processing CreateTablet for tablet d6e70ac8b41e4127881c5ce5167866de (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.215113 29329 tablet_service.cc:1467] Processing CreateTablet for tablet d6e70ac8b41e4127881c5ce5167866de (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.215217 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e70ac8b41e4127881c5ce5167866de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.215368 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e70ac8b41e4127881c5ce5167866de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.217712 2667 tablet_bootstrap.cc:492] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:00.214684 2666 tablet_bootstrap.cc:492] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:00.217939 29196 tablet_service.cc:1467] Processing CreateTablet for tablet d3d8471cf7654dbdab51d855ed30be18 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:00.218153 2668 tablet_bootstrap.cc:492] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:00.218231 2668 ts_tablet_manager.cc:1367] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:00.218233 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d8471cf7654dbdab51d855ed30be18. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.218698 2668 raft_consensus.cc:348] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.218797 2668 raft_consensus.cc:374] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.218824 2668 raft_consensus.cc:729] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.218935 2668 consensus_queue.cc:260] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.219251 2668 ts_tablet_manager.cc:1398] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.219704 2666 tablet_bootstrap.cc:654] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.219970 2668 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:00.220757 2668 tablet_bootstrap.cc:654] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.221029 2667 tablet_bootstrap.cc:654] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.221809 2671 tablet_bootstrap.cc:492] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:00.223193 2666 tablet_bootstrap.cc:492] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:00.223333 2666 ts_tablet_manager.cc:1367] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:59:00.223495 2668 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:00.223574 2668 ts_tablet_manager.cc:1367] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:00.223646 2669 tablet_bootstrap.cc:492] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:00.223984 2668 raft_consensus.cc:348] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.224010 2666 raft_consensus.cc:348] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.224081 2668 raft_consensus.cc:374] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.224089 2666 raft_consensus.cc:374] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.224109 2666 raft_consensus.cc:729] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.224108 2668 raft_consensus.cc:729] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.224195 2666 consensus_queue.cc:260] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.224220 2668 consensus_queue.cc:260] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.224452 2666 ts_tablet_manager.cc:1398] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:00.224476 2668 ts_tablet_manager.cc:1398] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.224856 2669 tablet_bootstrap.cc:654] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.225664 2672 tablet_bootstrap.cc:492] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:00.226469 2671 tablet_bootstrap.cc:654] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.226467 2675 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:00.226614 2672 tablet_bootstrap.cc:654] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.227360 2675 tablet_bootstrap.cc:654] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.227679 2672 tablet_bootstrap.cc:492] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:00.227759 2672 ts_tablet_manager.cc:1367] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:59:00.228160 2672 raft_consensus.cc:348] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.228255 2672 raft_consensus.cc:374] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.228281 2672 raft_consensus.cc:729] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.228399 2672 consensus_queue.cc:260] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.228657 2672 ts_tablet_manager.cc:1398] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:00.228745 2667 tablet_bootstrap.cc:492] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:00.228830 2667 ts_tablet_manager.cc:1367] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:00.229086 2669 tablet_bootstrap.cc:492] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:00.229152 2669 ts_tablet_manager.cc:1367] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:59:00.229254 2667 raft_consensus.cc:348] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.229348 2667 raft_consensus.cc:374] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.229378 2667 raft_consensus.cc:729] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.229483 2667 consensus_queue.cc:260] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.229589 2669 raft_consensus.cc:348] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.229671 2669 raft_consensus.cc:374] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.229696 2669 raft_consensus.cc:729] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.229748 2667 ts_tablet_manager.cc:1398] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.229801 2669 consensus_queue.cc:260] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.230079 2669 ts_tablet_manager.cc:1398] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.230229 2669 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:00.231040 2669 tablet_bootstrap.cc:654] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.233165 2675 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:00.233306 2675 ts_tablet_manager.cc:1367] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:00.233747 2675 raft_consensus.cc:348] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.233916 2675 raft_consensus.cc:374] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.233992 2675 raft_consensus.cc:729] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.234237 2675 consensus_queue.cc:260] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.234380 2671 tablet_bootstrap.cc:492] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:00.234464 2671 ts_tablet_manager.cc:1367] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:59:00.234881 2671 raft_consensus.cc:348] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.234975 2671 raft_consensus.cc:374] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.235002 2671 raft_consensus.cc:729] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.235103 2671 consensus_queue.cc:260] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.235224 2675 ts_tablet_manager.cc:1398] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:00.235383 2671 ts_tablet_manager.cc:1398] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.235471 2669 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:00.235544 2669 ts_tablet_manager.cc:1367] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:00.235944 2669 raft_consensus.cc:348] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.236018 2669 raft_consensus.cc:374] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.236037 2669 raft_consensus.cc:729] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.236127 2669 consensus_queue.cc:260] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.236362 2669 ts_tablet_manager.cc:1398] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.247169 1217 raft_consensus.cc:482] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:00.247258 1217 raft_consensus.cc:504] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.247596 1217 leader_election.cc:288] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:00.247994 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c528788d4f8a43a2a6614e7e9ef79fbd" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:00.248127 27063 raft_consensus.cc:2454] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:00.248154 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c528788d4f8a43a2a6614e7e9ef79fbd" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:00.248252 26793 raft_consensus.cc:2454] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:00.248435 26595 leader_election.cc:302] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:00.248665 1217 raft_consensus.cc:2781] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:00.248713 1217 raft_consensus.cc:482] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:00.248736 1217 raft_consensus.cc:3037] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.249573 1217 raft_consensus.cc:504] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.249897 1217 leader_election.cc:288] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:00.250301 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c528788d4f8a43a2a6614e7e9ef79fbd" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:00.250360 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c528788d4f8a43a2a6614e7e9ef79fbd" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:00.250411 27063 raft_consensus.cc:3037] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.250470 26793 raft_consensus.cc:3037] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.251258 27063 raft_consensus.cc:2454] T c528788d4f8a43a2a6614e7e9ef79fbd P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:00.251380 26793 raft_consensus.cc:2454] T c528788d4f8a43a2a6614e7e9ef79fbd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:00.251598 26595 leader_election.cc:302] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:00.251813 1217 raft_consensus.cc:2781] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:00.251962 1217 raft_consensus.cc:686] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:00.252087 1217 consensus_queue.cc:237] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.254089 26512 catalog_manager.cc:5484] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:00.278378 26512 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:00.278491 26512 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:00.281210 26512 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 497bba35b4e341a193b34c617a59e183
I20240502 01:59:00.281839 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet 497bba35b4e341a193b34c617a59e183 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:00 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:00.281847 29329 tablet_service.cc:1514] Processing DeleteTablet for tablet 497bba35b4e341a193b34c617a59e183 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:00 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:00.282318 2705 tablet_replica.cc:330] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:00.282371 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 497bba35b4e341a193b34c617a59e183 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:00 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:00.282459 2705 raft_consensus.cc:2229] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:00.282680 2705 raft_consensus.cc:2258] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.282948 2706 tablet_replica.cc:330] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:00.283130 2706 raft_consensus.cc:2229] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:00.283305 2706 raft_consensus.cc:2258] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.283995 2706 ts_tablet_manager.cc:1875] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:00.284389 2704 tablet_replica.cc:330] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:00.284564 2704 raft_consensus.cc:2229] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:00.284675 2704 raft_consensus.cc:2258] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.285214 2704 ts_tablet_manager.cc:1875] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:00.286708 2706 ts_tablet_manager.cc:1888] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:00.286782 2706 log.cc:1196] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/497bba35b4e341a193b34c617a59e183
I20240502 01:59:00.286975 2704 ts_tablet_manager.cc:1888] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:00.287091 2704 log.cc:1196] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/497bba35b4e341a193b34c617a59e183
I20240502 01:59:00.287194 2706 ts_tablet_manager.cc:1909] T 497bba35b4e341a193b34c617a59e183 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:00.287390 2704 ts_tablet_manager.cc:1909] T 497bba35b4e341a193b34c617a59e183 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:59:00.288008 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 497bba35b4e341a193b34c617a59e183 (table rebalancer_dynamic_table_new_name [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]) successfully deleted
I20240502 01:59:00.288070 2705 ts_tablet_manager.cc:1875] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:00.289757 2705 ts_tablet_manager.cc:1888] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:00.289841 2705 log.cc:1196] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/497bba35b4e341a193b34c617a59e183
I20240502 01:59:00.290282 2705 ts_tablet_manager.cc:1909] T 497bba35b4e341a193b34c617a59e183 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:00.298118 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 497bba35b4e341a193b34c617a59e183 (table rebalancer_dynamic_table_new_name [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]) successfully deleted
I20240502 01:59:00.298266 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 497bba35b4e341a193b34c617a59e183 (table rebalancer_dynamic_table_new_name [id=3b5f990c70ae416cb1f13f3f1a9cfe1a]) successfully deleted
I20240502 01:59:00.382622 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:00.387768 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 09c2103b7cb44a889a5f8ec63446e387 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]), partition=
I20240502 01:59:00.388037 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09c2103b7cb44a889a5f8ec63446e387. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.388289 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 09c2103b7cb44a889a5f8ec63446e387 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]), partition=
I20240502 01:59:00.388329 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 09c2103b7cb44a889a5f8ec63446e387 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]), partition=
I20240502 01:59:00.388494 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09c2103b7cb44a889a5f8ec63446e387. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.388537 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09c2103b7cb44a889a5f8ec63446e387. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:00.390532 2675 tablet_bootstrap.cc:492] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:00.390615 2667 tablet_bootstrap.cc:492] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:00.391433 2669 tablet_bootstrap.cc:492] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:00.391496 2667 tablet_bootstrap.cc:654] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.391530 2675 tablet_bootstrap.cc:654] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.392256 2669 tablet_bootstrap.cc:654] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:00.392870 2675 tablet_bootstrap.cc:492] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:00.392946 2675 ts_tablet_manager.cc:1367] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:00.393093 2667 tablet_bootstrap.cc:492] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:00.393175 2667 ts_tablet_manager.cc:1367] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:00.393332 2675 raft_consensus.cc:348] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.393420 2675 raft_consensus.cc:374] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.393448 2675 raft_consensus.cc:729] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.393546 2675 consensus_queue.cc:260] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.393615 2667 raft_consensus.cc:348] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.393707 2667 raft_consensus.cc:374] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.393738 2667 raft_consensus.cc:729] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.393760 2669 tablet_bootstrap.cc:492] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:00.393803 2675 ts_tablet_manager.cc:1398] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:00.393834 2669 ts_tablet_manager.cc:1367] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:00.393849 2667 consensus_queue.cc:260] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.394155 2667 ts_tablet_manager.cc:1398] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.394304 2669 raft_consensus.cc:348] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.394402 2669 raft_consensus.cc:374] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:00.394433 2669 raft_consensus.cc:729] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:00.394536 2669 consensus_queue.cc:260] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.394809 2669 ts_tablet_manager.cc:1398] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:00.414812 2528 raft_consensus.cc:482] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:00.414901 2528 raft_consensus.cc:504] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.415277 2528 leader_election.cc:288] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:00.415702 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d8471cf7654dbdab51d855ed30be18" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:00.415827 29213 raft_consensus.cc:2454] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:00.416085 26729 leader_election.cc:302] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:00.416199 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d8471cf7654dbdab51d855ed30be18" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:00.416301 29079 raft_consensus.cc:2454] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:00.416365 2226 raft_consensus.cc:2781] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:00.416420 2226 raft_consensus.cc:482] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:00.416460 2226 raft_consensus.cc:3037] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.417297 2226 raft_consensus.cc:504] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.417618 2226 leader_election.cc:288] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:00.417858 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d8471cf7654dbdab51d855ed30be18" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:00.417963 29213 raft_consensus.cc:3037] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.418155 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d8471cf7654dbdab51d855ed30be18" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:00.418258 29079 raft_consensus.cc:3037] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.418835 29213 raft_consensus.cc:2454] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:00.418947 29079 raft_consensus.cc:2454] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:00.419075 26729 leader_election.cc:302] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:00.419234 2226 raft_consensus.cc:2781] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:00.419312 2226 raft_consensus.cc:686] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:00.419425 2226 consensus_queue.cc:237] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.421145 26508 catalog_manager.cc:5484] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:00.477205 2472 raft_consensus.cc:482] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:00.477314 2472 raft_consensus.cc:504] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.477708 2472 leader_election.cc:288] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:00.478075 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e70ac8b41e4127881c5ce5167866de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:00.478085 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e70ac8b41e4127881c5ce5167866de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:00.478194 29079 raft_consensus.cc:2454] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:00.478206 29487 raft_consensus.cc:2454] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:00.478447 29283 leader_election.cc:302] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:00.478606 2472 raft_consensus.cc:2781] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:00.478662 2472 raft_consensus.cc:482] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:00.478706 2472 raft_consensus.cc:3037] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.479442 2472 raft_consensus.cc:504] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.479754 2472 leader_election.cc:288] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:00.480041 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e70ac8b41e4127881c5ce5167866de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:00.480041 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e70ac8b41e4127881c5ce5167866de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:00.480132 29079 raft_consensus.cc:3037] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.480167 29487 raft_consensus.cc:3037] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.480842 29079 raft_consensus.cc:2454] T d6e70ac8b41e4127881c5ce5167866de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:00.480877 29487 raft_consensus.cc:2454] T d6e70ac8b41e4127881c5ce5167866de P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:00.481026 2432 raft_consensus.cc:482] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:00.481083 29283 leader_election.cc:302] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:00.481102 2432 raft_consensus.cc:504] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.481248 2472 raft_consensus.cc:2781] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:00.481329 2472 raft_consensus.cc:686] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:00.481439 2432 leader_election.cc:288] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:00.481462 2472 consensus_queue.cc:237] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:00.482518 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09c2103b7cb44a889a5f8ec63446e387" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:00.482636 29213 raft_consensus.cc:2454] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:00.482740 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09c2103b7cb44a889a5f8ec63446e387" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:00.482849 26661 raft_consensus.cc:2454] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:00.483119 29015 leader_election.cc:302] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:00.483350 2432 raft_consensus.cc:2781] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:00.483402 2432 raft_consensus.cc:482] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:00.483438 2432 raft_consensus.cc:3037] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.483613 26508 catalog_manager.cc:5484] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:00.484136 2432 raft_consensus.cc:504] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.484465 2432 leader_election.cc:288] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:00.484813 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09c2103b7cb44a889a5f8ec63446e387" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:00.484850 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09c2103b7cb44a889a5f8ec63446e387" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:00.484921 29213 raft_consensus.cc:3037] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.484941 26661 raft_consensus.cc:3037] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:00.485854 26661 raft_consensus.cc:2454] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:00.485857 29213 raft_consensus.cc:2454] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:00.486097 29015 leader_election.cc:302] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:00.486384 2432 raft_consensus.cc:2781] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:00.486546 2432 raft_consensus.cc:686] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:00.486685 2432 consensus_queue.cc:237] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:00.488325 26508 catalog_manager.cc:5484] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:00.515144 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:00.515666 29196 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:59:00.518635 29329 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.523226 29454 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:00.530970 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:00.547570 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:00.551697 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.559062 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.592650 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:00.597097 26661 raft_consensus.cc:1261] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:00.597304 29213 raft_consensus.cc:1261] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:00.597796 2432 consensus_queue.cc:1035] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.598107 2432 consensus_queue.cc:1035] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.601434 26513 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]). Expected version 1 got 0
I20240502 01:59:00.602532 26508 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]). Expected version 1 got 0
I20240502 01:59:00.605584 26508 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]). Expected version 1 got 0
I20240502 01:59:00.606971 2544 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:00.607002 2734 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:00.607302 2544 tablet.cc:1620] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:00.607326 2734 tablet.cc:1620] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:00.607657 2740 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:00.607926 2740 tablet.cc:1620] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:00.608316 26513 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681]). Expected version 1 got 0
I20240502 01:59:00.613867 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681] alter complete (version 1)
I20240502 01:59:00.618959 29487 consensus_queue.cc:237] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:00.620070 26793 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:00.620389 26661 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:00.620450 1595 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:00.620761 2101 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:00.622653 1595 raft_consensus.cc:2932] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:00.623253 26793 raft_consensus.cc:2932] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:00.625487 26508 catalog_manager.cc:5484] T a326a7899c064c95b426aae61d65208f P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index 2 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New cstate: current_term: 2 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:00.627314 26661 raft_consensus.cc:2932] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:00.628098 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet a326a7899c064c95b426aae61d65208f with delete_type TABLET_DATA_TOMBSTONED (TS f3d18a019013487ab801599cfb80e4b4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:00.629119 2750 tablet_replica.cc:330] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:00.630069 2750 raft_consensus.cc:2229] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:00.630209 2750 raft_consensus.cc:2258] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.630573 2750 ts_tablet_manager.cc:1875] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:00.632894 2750 ts_tablet_manager.cc:1888] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240502 01:59:00.632956 2750 log.cc:1196] T a326a7899c064c95b426aae61d65208f P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/a326a7899c064c95b426aae61d65208f
I20240502 01:59:00.633451 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet a326a7899c064c95b426aae61d65208f (table rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]) successfully deleted
I20240502 01:59:00.654963 1217 consensus_queue.cc:1035] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.657605 2702 consensus_queue.cc:1035] T c528788d4f8a43a2a6614e7e9ef79fbd P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.670850 26794 consensus_queue.cc:237] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:00.672425 27063 raft_consensus.cc:1261] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:00.672453 26927 raft_consensus.cc:1261] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:59:00.672745 26730 consensus_peers.cc:485] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7b2aaf8eaa844c0390909ccfc0b4bb09. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:00.672961 2790 consensus_queue.cc:1035] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:00.673142 2528 consensus_queue.cc:1035] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:00.674881 2790 raft_consensus.cc:2932] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.675293 27063 raft_consensus.cc:2932] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.678292 26508 catalog_manager.cc:5484] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.678418 26927 raft_consensus.cc:2932] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.697954 26794 consensus_queue.cc:237] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:00.699209 29349 raft_consensus.cc:1261] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:00.699209 26927 raft_consensus.cc:1261] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:00.699748 26727 consensus_peers.cc:485] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bde4108189cc45a5a1e8ae2d0ef699fd. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:00.699752 2528 consensus_queue.cc:1035] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:00.699939 2226 consensus_queue.cc:1035] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:00.701958 2528 raft_consensus.cc:2932] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:00.702299 26927 raft_consensus.cc:2932] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:00.703833 29349 raft_consensus.cc:2932] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:00.705652 26507 catalog_manager.cc:5484] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:00.717290 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:00.722512 2734 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:00.722813 2734 tablet.cc:1620] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:00.723239 2740 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:00.723749 2740 tablet.cc:1620] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:00.725035 2544 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:00.725322 2544 tablet.cc:1620] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:00.728492 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=366090f0c5724ec4a753b0a22bf51681] alter complete (version 2)
I20240502 01:59:00.729384 26794 consensus_queue.cc:237] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:00.731105 27063 raft_consensus.cc:1261] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:00.731487 2226 consensus_queue.cc:1035] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:59:00.731835 26730 consensus_peers.cc:485] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 49ac3a00934944bdb5c9e440045532a3. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:00.732990 26659 raft_consensus.cc:1261] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:00.733402 2226 consensus_queue.cc:1035] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:00.735126 2528 raft_consensus.cc:2932] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.735543 26659 raft_consensus.cc:2932] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.736793 27063 raft_consensus.cc:2932] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.737782 26508 catalog_manager.cc:5484] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:00.755509 29349 consensus_queue.cc:237] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:00.756925 29216 raft_consensus.cc:1261] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:00.757048 26659 raft_consensus.cc:1261] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:00.757349 32364 consensus_queue.cc:1035] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:00.757860 29286 consensus_peers.cc:485] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01764f37959e4c6799e3a83f0d4ed2e1. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:00.758875 2847 consensus_queue.cc:1035] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:00.760906 32364 raft_consensus.cc:2932] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:00.761358 26659 raft_consensus.cc:2932] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:00.763240 29216 raft_consensus.cc:2932] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:00.764356 26507 catalog_manager.cc:5484] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:00.832720 2226 consensus_queue.cc:1035] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.836778 2793 consensus_queue.cc:1035] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.842986 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:00.848681 2734 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:00.849157 2740 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:00.854581 2544 tablet.cc:1722] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:00.856746 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=366090f0c5724ec4a753b0a22bf51681] alter complete (version 3)
I20240502 01:59:00.875844 29329 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.881659 29196 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:59:00.883561 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:00.890038 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.890601 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:00.893229 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:00.899019 29454 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:00.903172 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:00.913841 2847 consensus_queue.cc:1035] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.925014 32364 consensus_queue.cc:1035] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:00.958390 26509 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:00.958501 26509 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:00.960482 26509 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 09c2103b7cb44a889a5f8ec63446e387
I20240502 01:59:00.960945 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c2103b7cb44a889a5f8ec63446e387 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:00 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:00.961138 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c2103b7cb44a889a5f8ec63446e387 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:00 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:00.961141 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c2103b7cb44a889a5f8ec63446e387 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:00 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:00.961388 2750 tablet_replica.cc:330] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:00.961522 2750 raft_consensus.cc:2229] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:00.961719 2750 raft_consensus.cc:2258] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.962489 2750 ts_tablet_manager.cc:1875] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:00.964594 2750 ts_tablet_manager.cc:1888] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:00.964661 2750 log.cc:1196] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/09c2103b7cb44a889a5f8ec63446e387
I20240502 01:59:00.965042 2750 ts_tablet_manager.cc:1909] T 09c2103b7cb44a889a5f8ec63446e387 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:00.965657 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table_new_name [id=366090f0c5724ec4a753b0a22bf51681]) successfully deleted
I20240502 01:59:00.966183 2922 tablet_replica.cc:330] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:00.966368 2922 raft_consensus.cc:2229] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:00.966522 2922 raft_consensus.cc:2258] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.967068 2922 ts_tablet_manager.cc:1875] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:00.969204 2922 ts_tablet_manager.cc:1888] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:00.969316 2922 log.cc:1196] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/09c2103b7cb44a889a5f8ec63446e387
I20240502 01:59:00.969604 2922 ts_tablet_manager.cc:1909] T 09c2103b7cb44a889a5f8ec63446e387 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:00.970121 2923 tablet_replica.cc:330] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:00.970264 2923 raft_consensus.cc:2229] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:00.970283 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table_new_name [id=366090f0c5724ec4a753b0a22bf51681]) successfully deleted
I20240502 01:59:00.970398 2923 raft_consensus.cc:2258] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:00.970949 2923 ts_tablet_manager.cc:1875] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:00.972600 2923 ts_tablet_manager.cc:1888] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:00.972666 2923 log.cc:1196] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/09c2103b7cb44a889a5f8ec63446e387
I20240502 01:59:00.972915 2923 ts_tablet_manager.cc:1909] T 09c2103b7cb44a889a5f8ec63446e387 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:00.973474 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 09c2103b7cb44a889a5f8ec63446e387 (table rebalancer_dynamic_table_new_name [id=366090f0c5724ec4a753b0a22bf51681]) successfully deleted
I20240502 01:59:01.018716 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:01.026165 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca8563ff68c4fa590c17ef58e67afa0 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.026461 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca8563ff68c4fa590c17ef58e67afa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.027534 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 8640f1dd192e4c1f8d14aaa14737e4f7 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.027798 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8640f1dd192e4c1f8d14aaa14737e4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.028961 26908 tablet_service.cc:1467] Processing CreateTablet for tablet a87bdf7578c94ef8b28c5f79227f0af8 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.029199 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87bdf7578c94ef8b28c5f79227f0af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.029884 2955 tablet_bootstrap.cc:492] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:01.030990 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca8563ff68c4fa590c17ef58e67afa0 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.031136 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca8563ff68c4fa590c17ef58e67afa0 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.031258 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca8563ff68c4fa590c17ef58e67afa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.031340 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca8563ff68c4fa590c17ef58e67afa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.031631 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 8640f1dd192e4c1f8d14aaa14737e4f7 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.031889 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8640f1dd192e4c1f8d14aaa14737e4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.033433 2955 tablet_bootstrap.cc:654] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.033486 2957 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:01.033519 29329 tablet_service.cc:1467] Processing CreateTablet for tablet a87bdf7578c94ef8b28c5f79227f0af8 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.033783 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87bdf7578c94ef8b28c5f79227f0af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.033805 29468 tablet_service.cc:1467] Processing CreateTablet for tablet a87bdf7578c94ef8b28c5f79227f0af8 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.034062 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a87bdf7578c94ef8b28c5f79227f0af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.034612 2957 tablet_bootstrap.cc:654] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.035372 2958 tablet_bootstrap.cc:492] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:01.036343 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 8640f1dd192e4c1f8d14aaa14737e4f7 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.036547 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8640f1dd192e4c1f8d14aaa14737e4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.041738 2954 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.042672 2954 tablet_bootstrap.cc:654] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.042892 2957 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:01.042969 2957 ts_tablet_manager.cc:1367] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 01:59:01.043290 2956 tablet_bootstrap.cc:492] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:01.043457 2957 raft_consensus.cc:348] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.043551 2957 raft_consensus.cc:374] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.043581 2957 raft_consensus.cc:729] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.044071 2956 tablet_bootstrap.cc:654] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.045043 2956 tablet_bootstrap.cc:492] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:01.045125 2956 ts_tablet_manager.cc:1367] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:01.045235 2958 tablet_bootstrap.cc:654] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.045533 2956 raft_consensus.cc:348] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.045648 2956 raft_consensus.cc:374] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.045681 2956 raft_consensus.cc:729] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.045789 2956 consensus_queue.cc:260] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.046128 2956 ts_tablet_manager.cc:1398] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.046339 2957 consensus_queue.cc:260] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.047497 2957 ts_tablet_manager.cc:1398] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:01.047660 2957 tablet_bootstrap.cc:492] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:01.049350 2957 tablet_bootstrap.cc:654] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.052821 2958 tablet_bootstrap.cc:492] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:01.053853 2958 ts_tablet_manager.cc:1367] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 01:59:01.055137 2958 raft_consensus.cc:348] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.055323 2958 raft_consensus.cc:374] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.055660 2958 raft_consensus.cc:729] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.059845 2957 tablet_bootstrap.cc:492] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:01.059947 2957 ts_tablet_manager.cc:1367] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:59:01.056298 2958 consensus_queue.cc:260] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.061321 2958 ts_tablet_manager.cc:1398] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20240502 01:59:01.061708 29196 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:59:01.062810 2954 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:01.062897 2954 ts_tablet_manager.cc:1367] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 01:59:01.063372 2954 raft_consensus.cc:348] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.063467 2954 raft_consensus.cc:374] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.063495 2954 raft_consensus.cc:729] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.063621 2954 consensus_queue.cc:260] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.063791 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:01.063928 2954 ts_tablet_manager.cc:1398] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.063933 2957 raft_consensus.cc:348] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.064025 2957 raft_consensus.cc:374] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.064054 2957 raft_consensus.cc:729] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.064078 2954 tablet_bootstrap.cc:492] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.064208 2957 consensus_queue.cc:260] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.064497 2957 ts_tablet_manager.cc:1398] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:01.064931 2954 tablet_bootstrap.cc:654] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.067256 2432 raft_consensus.cc:482] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.067340 2432 raft_consensus.cc:504] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.067688 2432 leader_election.cc:288] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:01.068184 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8640f1dd192e4c1f8d14aaa14737e4f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:01.068295 29216 raft_consensus.cc:2454] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:01.068645 29015 leader_election.cc:302] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:01.069283 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8640f1dd192e4c1f8d14aaa14737e4f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:01.070166 2432 raft_consensus.cc:2781] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.070225 2432 raft_consensus.cc:482] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.070258 2432 raft_consensus.cc:3037] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
W20240502 01:59:01.070770 29013 leader_election.cc:341] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:01.071166 2432 raft_consensus.cc:504] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.071493 2432 leader_election.cc:288] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:01.071648 29329 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.071856 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8640f1dd192e4c1f8d14aaa14737e4f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
W20240502 01:59:01.072188 29013 leader_election.cc:341] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:01.072588 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8640f1dd192e4c1f8d14aaa14737e4f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:01.072685 29216 raft_consensus.cc:3037] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.073658 29216 raft_consensus.cc:2454] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:01.073928 29015 leader_election.cc:302] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters: 468dd27456c64d67b22e8a58af93d93d
I20240502 01:59:01.074575 2955 tablet_bootstrap.cc:492] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:01.074662 2955 ts_tablet_manager.cc:1367] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.045s user 0.001s sys 0.001s
I20240502 01:59:01.074911 2954 tablet_bootstrap.cc:492] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:01.074959 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 9bce310f4a4340a08150660227c46c73 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0]), partition=
I20240502 01:59:01.074982 2954 ts_tablet_manager.cc:1367] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:01.075152 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bce310f4a4340a08150660227c46c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.075090 2955 raft_consensus.cc:348] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.075177 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 9bce310f4a4340a08150660227c46c73 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0]), partition=
I20240502 01:59:01.075191 2955 raft_consensus.cc:374] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.075246 2955 raft_consensus.cc:729] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.075366 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bce310f4a4340a08150660227c46c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.075352 2955 consensus_queue.cc:260] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.075423 2954 raft_consensus.cc:348] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.075522 2954 raft_consensus.cc:374] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.075552 2954 raft_consensus.cc:729] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.075634 2955 ts_tablet_manager.cc:1398] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:01.075647 2954 consensus_queue.cc:260] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.075785 2955 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:01.075888 2954 ts_tablet_manager.cc:1398] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.076663 2955 tablet_bootstrap.cc:654] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.077178 2954 tablet_bootstrap.cc:492] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.077235 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 9bce310f4a4340a08150660227c46c73 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0]), partition=
I20240502 01:59:01.077445 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bce310f4a4340a08150660227c46c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.077940 2954 tablet_bootstrap.cc:654] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.079830 2432 raft_consensus.cc:2781] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.079917 2432 raft_consensus.cc:686] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:01.080034 2432 consensus_queue.cc:237] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.082780 26510 catalog_manager.cc:5484] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:01.083756 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.087965 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.089330 2957 tablet_bootstrap.cc:492] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:01.090210 29468 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:01.090318 2957 tablet_bootstrap.cc:654] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.091588 2979 tablet_bootstrap.cc:492] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:01.092203 2955 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:01.092288 2955 ts_tablet_manager.cc:1367] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240502 01:59:01.092696 2955 raft_consensus.cc:348] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.092790 2955 raft_consensus.cc:374] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.092815 2955 raft_consensus.cc:729] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.092916 2955 consensus_queue.cc:260] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.093207 2955 ts_tablet_manager.cc:1398] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:01.093358 2955 tablet_bootstrap.cc:492] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:01.094234 2955 tablet_bootstrap.cc:654] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.096516 2979 tablet_bootstrap.cc:654] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.099422 2954 tablet_bootstrap.cc:492] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:01.099524 2954 ts_tablet_manager.cc:1367] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 01:59:01.100001 2954 raft_consensus.cc:348] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.100131 2954 raft_consensus.cc:374] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.100186 2954 raft_consensus.cc:729] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.100315 2954 consensus_queue.cc:260] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.101202 2954 ts_tablet_manager.cc:1398] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:01.106396 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.106566 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:01.112071 2984 ts_tablet_manager.cc:902] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:01.121405 2984 tablet_copy_client.cc:308] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:01.121747 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:47995)
I20240502 01:59:01.121863 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:47995: session id = 7c693d0933c64d9babc07f404df45cd1-7b2aaf8eaa844c0390909ccfc0b4bb09
I20240502 01:59:01.122750 26815 tablet_copy_source_session.cc:205] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:01.123647 2984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.125730 2984 tablet_copy_client.cc:783] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:01.125864 2984 tablet_copy_client.cc:647] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:01.126447 1595 raft_consensus.cc:482] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.126524 1595 raft_consensus.cc:504] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.126848 1595 leader_election.cc:288] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:01.127065 2957 tablet_bootstrap.cc:492] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:01.127199 2957 ts_tablet_manager.cc:1367] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20240502 01:59:01.127526 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:01.127526 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:01.127632 29079 raft_consensus.cc:2454] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:01.127632 29349 raft_consensus.cc:2454] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:01.127660 2957 raft_consensus.cc:348] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.127765 2957 raft_consensus.cc:374] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.127799 2957 raft_consensus.cc:729] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.127911 2957 consensus_queue.cc:260] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.128098 29419 leader_election.cc:302] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:01.128165 2957 ts_tablet_manager.cc:1398] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.128329 1595 raft_consensus.cc:2781] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.128373 1595 raft_consensus.cc:482] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.128404 1595 raft_consensus.cc:3037] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.128671 2955 tablet_bootstrap.cc:492] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:01.128746 2955 ts_tablet_manager.cc:1367] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.000s
I20240502 01:59:01.129217 1595 raft_consensus.cc:504] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.129285 2955 raft_consensus.cc:348] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.129374 2955 raft_consensus.cc:374] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.129400 2955 raft_consensus.cc:729] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.129495 1595 leader_election.cc:288] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:01.129508 2955 consensus_queue.cc:260] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.129756 2955 ts_tablet_manager.cc:1398] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.129835 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:01.129922 29079 raft_consensus.cc:3037] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.131042 29079 raft_consensus.cc:2454] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:01.132512 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:01.132615 29349 raft_consensus.cc:3037] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.132895 29419 leader_election.cc:302] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:01.133098 1595 raft_consensus.cc:2781] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.133177 1595 raft_consensus.cc:686] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:01.133303 1595 consensus_queue.cc:237] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.133527 29349 raft_consensus.cc:2454] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:01.137290 26506 catalog_manager.cc:5484] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:01.136598 2979 tablet_bootstrap.cc:492] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:01.137678 2979 ts_tablet_manager.cc:1367] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20240502 01:59:01.138200 2979 raft_consensus.cc:348] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.138465 2979 raft_consensus.cc:374] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.138621 2979 raft_consensus.cc:729] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.138890 2979 consensus_queue.cc:260] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.140136 2984 tablet_copy_client.cc:523] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:01.140663 2979 ts_tablet_manager.cc:1398] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:01.141268 2984 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.150759 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.165390 2984 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:01.165622 2984 tablet_bootstrap.cc:492] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:59:01.165773 2984 ts_tablet_manager.cc:1367] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.025s user 0.019s sys 0.000s
I20240502 01:59:01.166190 2984 raft_consensus.cc:348] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:01.166294 2984 raft_consensus.cc:729] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.166414 2984 consensus_queue.cc:260] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:01.166689 2984 ts_tablet_manager.cc:1398] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 01:59:01.167178 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-7b2aaf8eaa844c0390909ccfc0b4bb09 received from {username='slave'} at 127.19.43.200:47995
I20240502 01:59:01.167259 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-7b2aaf8eaa844c0390909ccfc0b4bb09 on tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:01.174206 2992 ts_tablet_manager.cc:902] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:01.174710 2992 tablet_copy_client.cc:308] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:01.175346 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet bde4108189cc45a5a1e8ae2d0ef699fd from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:53523)
I20240502 01:59:01.175444 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet bde4108189cc45a5a1e8ae2d0ef699fd from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:53523: session id = f3d18a019013487ab801599cfb80e4b4-bde4108189cc45a5a1e8ae2d0ef699fd
I20240502 01:59:01.176201 26815 tablet_copy_source_session.cc:205] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:01.176945 2992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde4108189cc45a5a1e8ae2d0ef699fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.179309 2992 tablet_copy_client.cc:783] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:01.179453 2992 tablet_copy_client.cc:647] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:01.180641 2992 tablet_copy_client.cc:523] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:01.181747 2992 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:01.182868 32364 raft_consensus.cc:482] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.182956 32364 raft_consensus.cc:504] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.183315 32364 leader_election.cc:288] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:01.183724 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87bdf7578c94ef8b28c5f79227f0af8" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:01.183739 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87bdf7578c94ef8b28c5f79227f0af8" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:01.183843 26927 raft_consensus.cc:2454] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:01.183847 29487 raft_consensus.cc:2454] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:01.184087 2008 raft_consensus.cc:482] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.184166 2008 raft_consensus.cc:504] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.184530 2008 leader_election.cc:288] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:01.184916 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bce310f4a4340a08150660227c46c73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:01.185027 29079 raft_consensus.cc:2454] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:01.185218 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bce310f4a4340a08150660227c46c73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:01.185329 29487 raft_consensus.cc:2454] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:01.185439 2992 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:01.185525 26995 leader_election.cc:302] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:01.185678 2992 tablet_bootstrap.cc:492] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:59:01.185930 29286 leader_election.cc:302] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:01.185982 2992 ts_tablet_manager.cc:1367] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.001s
I20240502 01:59:01.186187 32364 raft_consensus.cc:2781] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.186235 32364 raft_consensus.cc:482] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.186268 32364 raft_consensus.cc:3037] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.186416 2992 raft_consensus.cc:348] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.186509 2992 raft_consensus.cc:729] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.186627 2992 consensus_queue.cc:260] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.187096 32364 raft_consensus.cc:504] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.187449 32364 leader_election.cc:288] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:01.187808 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87bdf7578c94ef8b28c5f79227f0af8" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:01.187806 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a87bdf7578c94ef8b28c5f79227f0af8" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:01.187897 29487 raft_consensus.cc:3037] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.187897 26927 raft_consensus.cc:3037] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.187597 2992 ts_tablet_manager.cc:1398] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:01.188745 29487 raft_consensus.cc:2454] T a87bdf7578c94ef8b28c5f79227f0af8 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:01.188746 26927 raft_consensus.cc:2454] T a87bdf7578c94ef8b28c5f79227f0af8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:01.189221 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-bde4108189cc45a5a1e8ae2d0ef699fd received from {username='slave'} at 127.19.43.197:53523
I20240502 01:59:01.189281 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-bde4108189cc45a5a1e8ae2d0ef699fd on tablet bde4108189cc45a5a1e8ae2d0ef699fd with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:01.189325 29286 leader_election.cc:302] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:01.190685 2008 raft_consensus.cc:2781] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.190742 2008 raft_consensus.cc:482] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.190773 2008 raft_consensus.cc:3037] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.191699 2008 raft_consensus.cc:504] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.192044 2008 leader_election.cc:288] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:01.192334 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bce310f4a4340a08150660227c46c73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:01.192425 29487 raft_consensus.cc:3037] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.193146 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bce310f4a4340a08150660227c46c73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:01.193240 29079 raft_consensus.cc:3037] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.193300 29487 raft_consensus.cc:2454] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:01.193533 26998 leader_election.cc:302] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:01.193676 1402 raft_consensus.cc:2781] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.193739 1402 raft_consensus.cc:686] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:01.193852 1402 consensus_queue.cc:237] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.194011 29079 raft_consensus.cc:2454] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:01.194386 32364 raft_consensus.cc:2781] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.194476 32364 raft_consensus.cc:686] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:01.194593 32364 consensus_queue.cc:237] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:01.195535 26508 catalog_manager.cc:5484] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.196455 26506 catalog_manager.cc:5484] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:01.217540 3011 ts_tablet_manager.cc:902] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:01.218072 3011 tablet_copy_client.cc:308] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.199:44303
I20240502 01:59:01.218850 29369 tablet_copy_service.cc:140] P 468dd27456c64d67b22e8a58af93d93d: Received BeginTabletCopySession request for tablet 01764f37959e4c6799e3a83f0d4ed2e1 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:43081)
I20240502 01:59:01.218971 29369 tablet_copy_service.cc:161] P 468dd27456c64d67b22e8a58af93d93d: Beginning new tablet copy session on tablet 01764f37959e4c6799e3a83f0d4ed2e1 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:43081: session id = 842c1b011b5d4c77bfd9b9591111502e-01764f37959e4c6799e3a83f0d4ed2e1
I20240502 01:59:01.220275 29369 tablet_copy_source_session.cc:205] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:01.221756 3011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01764f37959e4c6799e3a83f0d4ed2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.223803 3011 tablet_copy_client.cc:783] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:01.223937 3011 tablet_copy_client.cc:647] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:01.225219 3011 tablet_copy_client.cc:523] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:01.226380 3011 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:01.229401 3011 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:01.229631 3011 tablet_bootstrap.cc:492] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:01.229780 3011 ts_tablet_manager.cc:1367] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:01.230181 3011 raft_consensus.cc:348] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:01.230289 3011 raft_consensus.cc:729] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.230405 3011 consensus_queue.cc:260] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:01.230683 3011 ts_tablet_manager.cc:1398] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:59:01.231472 29369 tablet_copy_service.cc:342] P 468dd27456c64d67b22e8a58af93d93d: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-01764f37959e4c6799e3a83f0d4ed2e1 received from {username='slave'} at 127.19.43.195:43081
I20240502 01:59:01.231557 29369 tablet_copy_service.cc:433] P 468dd27456c64d67b22e8a58af93d93d: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-01764f37959e4c6799e3a83f0d4ed2e1 on tablet 01764f37959e4c6799e3a83f0d4ed2e1 with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:01.288321 2984 ts_tablet_manager.cc:902] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:01.288748 2984 tablet_copy_client.cc:308] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:01.289227 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 49ac3a00934944bdb5c9e440045532a3 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:47995)
I20240502 01:59:01.289326 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 49ac3a00934944bdb5c9e440045532a3 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:47995: session id = 7c693d0933c64d9babc07f404df45cd1-49ac3a00934944bdb5c9e440045532a3
I20240502 01:59:01.290781 26815 tablet_copy_source_session.cc:205] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:01.291924 2984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49ac3a00934944bdb5c9e440045532a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.293524 2984 tablet_copy_client.cc:783] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:01.293660 2984 tablet_copy_client.cc:647] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:01.295120 2984 tablet_copy_client.cc:523] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:01.296358 2984 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.303288 27044 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.304307 29468 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.306934 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:01.309231 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.311026 29196 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:59:01.315479 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:01.321825 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.325863 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.349340 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.355392 29079 raft_consensus.cc:1261] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:01.355820 2008 consensus_queue.cc:1035] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.356190 2850 raft_consensus.cc:482] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.356280 2850 raft_consensus.cc:504] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.356750 2850 leader_election.cc:288] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:01.357268 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8640f1dd192e4c1f8d14aaa14737e4f7" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:01.357933 29487 raft_consensus.cc:1261] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:01.358660 2008 consensus_queue.cc:1035] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.358969 26507 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 9bce310f4a4340a08150660227c46c73 (table rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0]). Expected version 1 got 0
I20240502 01:59:01.360196 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8640f1dd192e4c1f8d14aaa14737e4f7" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:01.360370 29213 raft_consensus.cc:2379] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 468dd27456c64d67b22e8a58af93d93d in current term 1: Already voted for candidate f3d18a019013487ab801599cfb80e4b4 in this term.
I20240502 01:59:01.361263 3067 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:01.361443 29285 leader_election.cc:302] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:01.361573 26507 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 9bce310f4a4340a08150660227c46c73 (table rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0]). Expected version 1 got 0
I20240502 01:59:01.361713 3067 tablet.cc:1620] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:01.361754 2850 raft_consensus.cc:3037] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.362354 3070 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:01.362690 3070 tablet.cc:1620] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:01.363752 26507 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 9bce310f4a4340a08150660227c46c73 (table rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0]). Expected version 1 got 0
I20240502 01:59:01.364567 2850 raft_consensus.cc:2726] T 8640f1dd192e4c1f8d14aaa14737e4f7 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 01:59:01.367563 3069 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:01.368269 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0] alter complete (version 1)
I20240502 01:59:01.370412 3069 tablet.cc:1620] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:01.384119 2984 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:01.384372 2984 tablet_bootstrap.cc:492] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:59:01.384526 2984 ts_tablet_manager.cc:1367] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.088s user 0.022s sys 0.000s
I20240502 01:59:01.384938 2984 raft_consensus.cc:348] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:01.385113 2984 raft_consensus.cc:729] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.385254 2984 consensus_queue.cc:260] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:01.385620 2984 ts_tablet_manager.cc:1398] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:59:01.386076 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-49ac3a00934944bdb5c9e440045532a3 received from {username='slave'} at 127.19.43.200:47995
I20240502 01:59:01.386152 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-49ac3a00934944bdb5c9e440045532a3 on tablet 49ac3a00934944bdb5c9e440045532a3 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:01.479478 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:01.485569 29468 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.486506 3067 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:01.486827 3067 tablet.cc:1620] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:01.487722 3069 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:01.487991 3069 tablet.cc:1620] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:01.491446 3070 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:01.491722 3070 tablet.cc:1620] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:01.496093 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.498150 29196 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:59:01.498425 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=4100529fd7c840e9b72c0984ecca91a0] alter complete (version 2)
I20240502 01:59:01.498842 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.507768 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.508123 29487 raft_consensus.cc:1203] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:59:01.515933 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.518110 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:01.546260 27044 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.553112 2989 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.561014 1595 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.592530 26927 consensus_queue.cc:237] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.593644 29349 raft_consensus.cc:1261] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:01.593663 26794 raft_consensus.cc:1261] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:01.594090 3124 consensus_queue.cc:1035] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:01.594297 2817 consensus_queue.cc:1035] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:01.596009 2817 raft_consensus.cc:2932] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:01.598981 26794 raft_consensus.cc:2932] T d14b1389d460401b83d3652b25aaddec P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:01.599560 26506 catalog_manager.cc:5484] T d14b1389d460401b83d3652b25aaddec P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:01.600251 29349 raft_consensus.cc:2932] T d14b1389d460401b83d3652b25aaddec P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:01.603116 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet d14b1389d460401b83d3652b25aaddec with delete_type TABLET_DATA_TOMBSTONED (TS 0d83fdd1e6e04cf5a5602994d1dbc0df not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:01.603425 3149 tablet_replica.cc:330] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:01.603644 3149 raft_consensus.cc:2229] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:01.603770 3149 raft_consensus.cc:2258] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:01.604223 3149 ts_tablet_manager.cc:1875] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:01.606568 3149 ts_tablet_manager.cc:1888] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:01.606694 3149 log.cc:1196] T d14b1389d460401b83d3652b25aaddec P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/d14b1389d460401b83d3652b25aaddec
I20240502 01:59:01.606966 2432 consensus_queue.cc:1035] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.607331 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet d14b1389d460401b83d3652b25aaddec (table rebalancer_extra_table_4 [id=8a5690e9c5f248b79f643601d74417cd]) successfully deleted
I20240502 01:59:01.609200 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:01.610709 2712 consensus_queue.cc:1035] T 8640f1dd192e4c1f8d14aaa14737e4f7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.615973 3067 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:01.616955 3069 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:01.619721 3070 tablet.cc:1722] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:01.623888 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=4100529fd7c840e9b72c0984ecca91a0] alter complete (version 3)
I20240502 01:59:01.669726 32364 consensus_queue.cc:1035] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.676286 32364 consensus_queue.cc:1035] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:01.682572 29080 raft_consensus.cc:1203] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:01.718820 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:01.720371 29468 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.720976 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:01.731712 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:01.731834 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:01.732390 29454 tablet_service.cc:1467] Processing CreateTablet for tablet 713002b695184076816d7748c052ce59 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.732657 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 713002b695184076816d7748c052ce59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.733362 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 071041f8947347aa81e02721ed08731b (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.733433 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 713002b695184076816d7748c052ce59 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.733621 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071041f8947347aa81e02721ed08731b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.733733 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 713002b695184076816d7748c052ce59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.735029 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 071041f8947347aa81e02721ed08731b (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.735272 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071041f8947347aa81e02721ed08731b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.736389 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 8209cefcc49042b18fdd97d845d167e2 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.736634 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8209cefcc49042b18fdd97d845d167e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.736629 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 713002b695184076816d7748c052ce59 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.736822 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 713002b695184076816d7748c052ce59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.739140 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 8209cefcc49042b18fdd97d845d167e2 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.739251 3217 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:01.739374 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8209cefcc49042b18fdd97d845d167e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.739823 3214 tablet_bootstrap.cc:492] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.740283 3217 tablet_bootstrap.cc:654] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.740597 3216 tablet_bootstrap.cc:492] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:01.741000 3215 tablet_bootstrap.cc:492] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:01.741220 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.741986 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 071041f8947347aa81e02721ed08731b (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.742251 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071041f8947347aa81e02721ed08731b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.742650 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 9bce310f4a4340a08150660227c46c73
I20240502 01:59:01.743304 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bce310f4a4340a08150660227c46c73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:01 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:01.743891 27044 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:01.741946 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 8209cefcc49042b18fdd97d845d167e2 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:01.744736 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8209cefcc49042b18fdd97d845d167e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.745432 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bce310f4a4340a08150660227c46c73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:01 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:01.745786 3216 tablet_bootstrap.cc:654] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.746721 3215 tablet_bootstrap.cc:654] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.747223 3214 tablet_bootstrap.cc:654] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.748261 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bce310f4a4340a08150660227c46c73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:01 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:01.753397 3221 tablet_bootstrap.cc:492] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:01.753397 29196 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 01:59:01.753592 3222 tablet_replica.cc:330] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:01.753758 3222 raft_consensus.cc:2229] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:01.754396 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.754678 3222 raft_consensus.cc:2258] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:01.755226 3222 ts_tablet_manager.cc:1875] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:01.758196 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:01.759750 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:01.759876 3221 tablet_bootstrap.cc:654] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.761093 3217 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:01.761183 3217 ts_tablet_manager.cc:1367] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 01:59:01.761621 3217 raft_consensus.cc:348] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.761782 3217 raft_consensus.cc:374] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.761813 3217 raft_consensus.cc:729] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.761916 3217 consensus_queue.cc:260] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.762333 3217 ts_tablet_manager.cc:1398] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.762538 3225 tablet_replica.cc:330] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:01.762836 3225 raft_consensus.cc:2229] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:01.763029 3225 raft_consensus.cc:2258] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:01.763499 3225 ts_tablet_manager.cc:1875] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:01.766316 3225 ts_tablet_manager.cc:1888] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:01.766418 3225 log.cc:1196] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/9bce310f4a4340a08150660227c46c73
I20240502 01:59:01.766755 3225 ts_tablet_manager.cc:1909] T 9bce310f4a4340a08150660227c46c73 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:01.767355 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 9bce310f4a4340a08150660227c46c73 (table rebalancer_dynamic_table_new_name [id=4100529fd7c840e9b72c0984ecca91a0]) successfully deleted
I20240502 01:59:01.779382 3222 ts_tablet_manager.cc:1888] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:01.779449 3222 log.cc:1196] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/9bce310f4a4340a08150660227c46c73
I20240502 01:59:01.779678 3222 ts_tablet_manager.cc:1909] T 9bce310f4a4340a08150660227c46c73 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:59:01.780400 3215 tablet_bootstrap.cc:492] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:01.781522 3215 ts_tablet_manager.cc:1367] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240502 01:59:01.782055 3215 raft_consensus.cc:348] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.782348 3218 tablet_bootstrap.cc:492] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:01.783313 3218 tablet_bootstrap.cc:654] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.783408 3215 raft_consensus.cc:374] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.783483 3215 raft_consensus.cc:729] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.783638 3215 consensus_queue.cc:260] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.783990 3215 ts_tablet_manager.cc:1398] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:01.787791 3215 tablet_bootstrap.cc:492] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:01.789104 3215 tablet_bootstrap.cc:654] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.790295 3220 tablet_replica.cc:330] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:01.790374 3215 tablet_bootstrap.cc:492] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:01.790481 3220 raft_consensus.cc:2229] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:01.790522 3215 ts_tablet_manager.cc:1367] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:01.790669 3220 raft_consensus.cc:2258] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:01.791069 3215 raft_consensus.cc:348] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.791211 3215 raft_consensus.cc:374] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.791267 3215 raft_consensus.cc:729] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.791297 3220 ts_tablet_manager.cc:1875] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:01.791438 3215 consensus_queue.cc:260] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.791832 3215 ts_tablet_manager.cc:1398] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.797329 3220 ts_tablet_manager.cc:1888] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:01.797420 3220 log.cc:1196] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/9bce310f4a4340a08150660227c46c73
I20240502 01:59:01.797744 3220 ts_tablet_manager.cc:1909] T 9bce310f4a4340a08150660227c46c73 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:01.798427 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 9bce310f4a4340a08150660227c46c73 (table rebalancer_dynamic_table_new_name [id=4100529fd7c840e9b72c0984ecca91a0]) successfully deleted
I20240502 01:59:01.798708 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 9bce310f4a4340a08150660227c46c73 (table rebalancer_dynamic_table_new_name [id=4100529fd7c840e9b72c0984ecca91a0]) successfully deleted
I20240502 01:59:01.799413 3218 tablet_bootstrap.cc:492] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:01.799501 3218 ts_tablet_manager.cc:1367] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:59:01.800083 3218 raft_consensus.cc:348] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.800213 3218 raft_consensus.cc:374] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.800338 3218 raft_consensus.cc:729] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.800500 3218 consensus_queue.cc:260] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.800877 3218 ts_tablet_manager.cc:1398] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.805961 3216 tablet_bootstrap.cc:492] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:01.806085 3216 ts_tablet_manager.cc:1367] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.000s
I20240502 01:59:01.806594 3216 raft_consensus.cc:348] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.806686 3216 raft_consensus.cc:374] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.806715 3216 raft_consensus.cc:729] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.806881 3216 consensus_queue.cc:260] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.807178 3216 ts_tablet_manager.cc:1398] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.812137 3221 tablet_bootstrap.cc:492] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:01.812250 3221 ts_tablet_manager.cc:1367] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20240502 01:59:01.812544 26930 raft_consensus.cc:1203] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:01.812666 3221 raft_consensus.cc:348] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.812757 3221 raft_consensus.cc:374] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.812781 3221 raft_consensus.cc:729] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.812894 3221 consensus_queue.cc:260] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.813129 3221 ts_tablet_manager.cc:1398] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.813292 3221 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:01.814237 3221 tablet_bootstrap.cc:654] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.815196 3221 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:01.815291 3221 ts_tablet_manager.cc:1367] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:01.815660 3221 raft_consensus.cc:348] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.815747 3221 raft_consensus.cc:374] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.815778 3221 raft_consensus.cc:729] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.815892 3221 consensus_queue.cc:260] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.816138 3221 ts_tablet_manager.cc:1398] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.818181 3214 tablet_bootstrap.cc:492] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:01.818305 3214 ts_tablet_manager.cc:1367] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.000s
I20240502 01:59:01.818790 3214 raft_consensus.cc:348] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.818899 3214 raft_consensus.cc:374] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.818931 3214 raft_consensus.cc:729] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.819047 3214 consensus_queue.cc:260] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.819306 3214 ts_tablet_manager.cc:1398] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.819455 3214 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:01.820326 3214 tablet_bootstrap.cc:654] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.822202 3214 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:01.822275 3214 ts_tablet_manager.cc:1367] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:01.822638 3214 raft_consensus.cc:348] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.822726 3214 raft_consensus.cc:374] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.822753 3214 raft_consensus.cc:729] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.822857 3214 consensus_queue.cc:260] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.823097 3214 ts_tablet_manager.cc:1398] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.845840 26514 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:01.846450 29487 raft_consensus.cc:1203] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:59:01.851960 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 401be52e18ca4845a92f5215163a748b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe]), partition=
I20240502 01:59:01.852195 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401be52e18ca4845a92f5215163a748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.852830 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 401be52e18ca4845a92f5215163a748b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe]), partition=
I20240502 01:59:01.852917 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 401be52e18ca4845a92f5215163a748b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe]), partition=
I20240502 01:59:01.853047 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401be52e18ca4845a92f5215163a748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.853123 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 401be52e18ca4845a92f5215163a748b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:01.854342 3217 tablet_bootstrap.cc:492] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:01.854939 3218 tablet_bootstrap.cc:492] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:01.855206 3217 tablet_bootstrap.cc:654] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.855463 3215 tablet_bootstrap.cc:492] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:01.855726 3218 tablet_bootstrap.cc:654] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.856256 3217 tablet_bootstrap.cc:492] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:01.856331 3217 ts_tablet_manager.cc:1367] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:01.856338 3215 tablet_bootstrap.cc:654] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:01.856724 3217 raft_consensus.cc:348] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.856812 3217 raft_consensus.cc:374] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.856838 3217 raft_consensus.cc:729] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.856945 3217 consensus_queue.cc:260] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.857215 3217 ts_tablet_manager.cc:1398] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.857338 3215 tablet_bootstrap.cc:492] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:01.857409 3215 ts_tablet_manager.cc:1367] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:01.857595 3218 tablet_bootstrap.cc:492] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:01.857668 3218 ts_tablet_manager.cc:1367] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:01.857589 26793 consensus_queue.cc:237] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.857812 3215 raft_consensus.cc:348] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.857929 3215 raft_consensus.cc:374] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.857960 3215 raft_consensus.cc:729] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.858089 3215 consensus_queue.cc:260] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.858126 3218 raft_consensus.cc:348] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.858230 3218 raft_consensus.cc:374] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:01.858259 3218 raft_consensus.cc:729] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:01.858362 3215 ts_tablet_manager.cc:1398] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.858369 3218 consensus_queue.cc:260] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.858664 3218 ts_tablet_manager.cc:1398] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:01.859248 29349 raft_consensus.cc:1261] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:01.859339 29080 raft_consensus.cc:1261] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:01.859757 2528 consensus_queue.cc:1035] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:01.861881 3252 consensus_queue.cc:1035] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20240502 01:59:01.862236 3252 raft_consensus.cc:2932] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.865134 26514 catalog_manager.cc:5484] T bde4108189cc45a5a1e8ae2d0ef699fd P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 2 to 3, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.865950 29349 raft_consensus.cc:2932] T bde4108189cc45a5a1e8ae2d0ef699fd P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.866695 29079 raft_consensus.cc:2932] T bde4108189cc45a5a1e8ae2d0ef699fd P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.869227 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet bde4108189cc45a5a1e8ae2d0ef699fd with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:01.869581 3261 tablet_replica.cc:330] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:01.870487 3261 raft_consensus.cc:2229] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:01.870638 3261 raft_consensus.cc:2258] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:01.871260 3261 ts_tablet_manager.cc:1875] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:01.874320 3261 ts_tablet_manager.cc:1888] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:01.874698 3261 log.cc:1196] T bde4108189cc45a5a1e8ae2d0ef699fd P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/bde4108189cc45a5a1e8ae2d0ef699fd
I20240502 01:59:01.875397 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet bde4108189cc45a5a1e8ae2d0ef699fd (table rebalancer_extra_table_10 [id=5942d660595e4276b446bb801e4ba849]) successfully deleted
I20240502 01:59:01.886940 1595 raft_consensus.cc:482] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.887027 1595 raft_consensus.cc:504] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.887382 1595 leader_election.cc:288] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:01.887699 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:01.887820 29349 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:01.888031 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:01.888100 29419 leader_election.cc:302] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:01.888170 26659 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:01.888249 1595 raft_consensus.cc:2781] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.888316 1595 raft_consensus.cc:482] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.888365 1595 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.889180 1595 raft_consensus.cc:504] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.889490 1595 leader_election.cc:288] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:01.889801 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:01.889900 29349 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.889974 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:01.890106 26659 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.890820 29349 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:01.891248 26659 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:01.891533 29421 leader_election.cc:302] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:01.891700 1595 raft_consensus.cc:2781] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.891794 1595 raft_consensus.cc:686] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:01.891937 1595 consensus_queue.cc:237] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:01.894078 26514 catalog_manager.cc:5484] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:01.900198 26793 consensus_queue.cc:237] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:01.901379 26659 raft_consensus.cc:1261] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:01.901436 27064 raft_consensus.cc:1261] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:59:01.901697 26730 consensus_peers.cc:485] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 052566382186437e8e0245de9f60c30c. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:01.901741 3252 consensus_queue.cc:1035] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:01.901895 2528 consensus_queue.cc:1035] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:01.903745 3252 raft_consensus.cc:2932] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:01.904228 27063 raft_consensus.cc:2932] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:01.907581 26514 catalog_manager.cc:5484] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:01.910233 26659 raft_consensus.cc:2932] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:01.916249 2910 raft_consensus.cc:482] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.916349 2910 raft_consensus.cc:504] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.916787 2910 leader_election.cc:288] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:01.917019 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401be52e18ca4845a92f5215163a748b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:01.917075 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401be52e18ca4845a92f5215163a748b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:01.917135 29079 raft_consensus.cc:2454] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:01.917178 29349 raft_consensus.cc:2454] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:01.917424 29147 leader_election.cc:302] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:01.917598 2910 raft_consensus.cc:2781] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.917655 2910 raft_consensus.cc:482] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.917698 2910 raft_consensus.cc:3037] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.918576 2910 raft_consensus.cc:504] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.918898 2910 leader_election.cc:288] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:01.919256 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401be52e18ca4845a92f5215163a748b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:01.919291 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "401be52e18ca4845a92f5215163a748b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:01.919358 29349 raft_consensus.cc:3037] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.919389 29079 raft_consensus.cc:3037] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.920159 29349 raft_consensus.cc:2454] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:01.920195 29079 raft_consensus.cc:2454] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:01.920424 29147 leader_election.cc:302] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:01.920573 2910 raft_consensus.cc:2781] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.920737 2910 raft_consensus.cc:686] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:01.920876 2910 consensus_queue.cc:237] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:01.922789 26514 catalog_manager.cc:5484] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:01.937202 26661 consensus_queue.cc:237] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
W20240502 01:59:01.938709 26592 consensus_peers.cc:485] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69df2cdb31d640ff9f43bb395f2f26fe. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:01.938871 26930 raft_consensus.cc:1261] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:01.939002 26793 raft_consensus.cc:1261] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:01.939420 3305 consensus_queue.cc:1035] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:01.939656 3306 consensus_queue.cc:1035] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:01.946722 3306 raft_consensus.cc:2932] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.947160 26930 raft_consensus.cc:2932] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.949630 26514 catalog_manager.cc:5484] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.950630 26793 raft_consensus.cc:2932] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.958175 2101 raft_consensus.cc:482] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.958259 2101 raft_consensus.cc:504] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.958613 2101 leader_election.cc:288] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:01.958969 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:01.959093 29213 raft_consensus.cc:2454] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:01.959336 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:01.959360 29421 leader_election.cc:302] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:01.959445 29079 raft_consensus.cc:2454] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:01.959565 2101 raft_consensus.cc:2781] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.959626 2101 raft_consensus.cc:482] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.959662 2101 raft_consensus.cc:3037] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.960383 2101 raft_consensus.cc:504] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.960702 2101 leader_election.cc:288] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:01.960995 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:01.961093 29213 raft_consensus.cc:3037] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.961094 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:01.961201 29079 raft_consensus.cc:3037] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.962007 29213 raft_consensus.cc:2454] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:01.962311 29421 leader_election.cc:302] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:01.962466 2101 raft_consensus.cc:2781] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.962533 2101 raft_consensus.cc:686] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:01.962602 29079 raft_consensus.cc:2454] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:01.962648 2101 consensus_queue.cc:237] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:01.964725 26514 catalog_manager.cc:5484] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:01.992826 2910 raft_consensus.cc:482] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:01.992913 2910 raft_consensus.cc:504] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.993345 2910 leader_election.cc:288] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:01.994114 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071041f8947347aa81e02721ed08731b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:01.994215 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071041f8947347aa81e02721ed08731b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:01.994246 26661 raft_consensus.cc:2454] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:01.994320 26793 raft_consensus.cc:2454] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:01.994621 29149 leader_election.cc:302] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:01.994812 2910 raft_consensus.cc:2781] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:01.994864 2910 raft_consensus.cc:482] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:01.994899 2910 raft_consensus.cc:3037] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.995921 2910 raft_consensus.cc:504] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:01.996273 2910 leader_election.cc:288] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:01.996610 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071041f8947347aa81e02721ed08731b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:01.996678 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071041f8947347aa81e02721ed08731b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:01.996726 26661 raft_consensus.cc:3037] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.996764 26793 raft_consensus.cc:3037] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:01.997608 26793 raft_consensus.cc:2454] T 071041f8947347aa81e02721ed08731b P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:01.997628 26661 raft_consensus.cc:2454] T 071041f8947347aa81e02721ed08731b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:01.997892 29149 leader_election.cc:302] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:01.998100 2910 raft_consensus.cc:2781] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:01.998188 2910 raft_consensus.cc:686] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:01.998322 2910 consensus_queue.cc:237] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.000720 26514 catalog_manager.cc:5484] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:02.040472 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:02.044790 29349 raft_consensus.cc:1261] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:02.044802 29079 raft_consensus.cc:1261] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:02.045179 2910 consensus_queue.cc:1035] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.045358 3297 consensus_queue.cc:1035] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.048312 3354 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:02.049152 26509 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 401be52e18ca4845a92f5215163a748b (table rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe]). Expected version 1 got 0
I20240502 01:59:02.049485 3070 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:02.049767 3070 tablet.cc:1620] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:02.049975 26512 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 401be52e18ca4845a92f5215163a748b (table rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe]). Expected version 1 got 0
I20240502 01:59:02.052136 3354 tablet.cc:1620] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:02.054509 26512 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 401be52e18ca4845a92f5215163a748b (table rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe]). Expected version 1 got 0
I20240502 01:59:02.055138 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe] alter complete (version 1)
I20240502 01:59:02.058986 3355 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:02.060343 3355 tablet.cc:1620] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:02.067646 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:02.071097 29196 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.074173 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:02.075932 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:02.086167 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.086581 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:02.093288 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:02.099143 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.159194 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:02.165186 3354 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:02.165513 3354 tablet.cc:1620] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:02.165753 3070 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:02.166061 3355 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:02.166108 3070 tablet.cc:1620] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:02.166337 3355 tablet.cc:1620] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:02.171140 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=6303650c8b074e7585874da5709d09fe] alter complete (version 2)
I20240502 01:59:02.237955 29196 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.243017 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.244999 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:02.245579 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.245646 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:02.247009 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:02.252113 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:02.255329 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:02.278596 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:02.292165 3354 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:02.295471 3355 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:02.298393 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=6303650c8b074e7585874da5709d09fe] alter complete (version 3)
I20240502 01:59:02.299064 3070 tablet.cc:1722] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:02.327286 1595 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.332208 2101 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.349073 3438 ts_tablet_manager.cc:902] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:02.349566 3438 tablet_copy_client.cc:308] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:59:02.350000 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet 69df2cdb31d640ff9f43bb395f2f26fe from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:39103)
I20240502 01:59:02.350117 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet 69df2cdb31d640ff9f43bb395f2f26fe from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:39103: session id = f3d18a019013487ab801599cfb80e4b4-69df2cdb31d640ff9f43bb395f2f26fe
I20240502 01:59:02.351094 26682 tablet_copy_source_session.cc:205] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:02.351943 3438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69df2cdb31d640ff9f43bb395f2f26fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.354456 3438 tablet_copy_client.cc:783] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:02.354658 3438 tablet_copy_client.cc:647] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:02.355813 3438 tablet_copy_client.cc:523] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:02.357321 3438 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:02.392545 3438 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:02.392850 3438 tablet_bootstrap.cc:492] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:59:02.393040 3438 ts_tablet_manager.cc:1367] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.036s user 0.022s sys 0.000s
I20240502 01:59:02.393373 2101 consensus_queue.cc:1035] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.393539 3438 raft_consensus.cc:348] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.393654 3438 raft_consensus.cc:729] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.393783 3438 consensus_queue.cc:260] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.394110 3438 ts_tablet_manager.cc:1398] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 01:59:02.394799 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-69df2cdb31d640ff9f43bb395f2f26fe received from {username='slave'} at 127.19.43.197:39103
I20240502 01:59:02.394903 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-69df2cdb31d640ff9f43bb395f2f26fe on tablet 69df2cdb31d640ff9f43bb395f2f26fe with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:02.396889 1595 consensus_queue.cc:1035] T 713002b695184076816d7748c052ce59 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.403923 26509 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:02.404047 26509 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:02.406311 26509 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 401be52e18ca4845a92f5215163a748b
I20240502 01:59:02.406836 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 401be52e18ca4845a92f5215163a748b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:02 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:02.406997 29329 tablet_service.cc:1514] Processing DeleteTablet for tablet 401be52e18ca4845a92f5215163a748b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:02 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:02.407325 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet 401be52e18ca4845a92f5215163a748b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:02 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:02.409034 3478 tablet_replica.cc:330] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:02.409188 3478 raft_consensus.cc:2229] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:02.409356 3478 raft_consensus.cc:2258] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:02.410792 3478 ts_tablet_manager.cc:1875] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:02.412636 3476 tablet_replica.cc:330] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:02.412832 3476 raft_consensus.cc:2229] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:02.413000 3476 raft_consensus.cc:2258] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:02.413264 3477 tablet_replica.cc:330] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:02.413645 3476 ts_tablet_manager.cc:1875] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:02.413859 3477 raft_consensus.cc:2229] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:02.413920 3478 ts_tablet_manager.cc:1888] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:02.414044 3478 log.cc:1196] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/401be52e18ca4845a92f5215163a748b
I20240502 01:59:02.414052 3477 raft_consensus.cc:2258] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:02.414403 3478 ts_tablet_manager.cc:1909] T 401be52e18ca4845a92f5215163a748b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:02.415103 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 401be52e18ca4845a92f5215163a748b (table rebalancer_dynamic_table_new_name [id=6303650c8b074e7585874da5709d09fe]) successfully deleted
I20240502 01:59:02.416064 3476 ts_tablet_manager.cc:1888] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:02.416131 3476 log.cc:1196] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/401be52e18ca4845a92f5215163a748b
I20240502 01:59:02.416332 3476 ts_tablet_manager.cc:1909] T 401be52e18ca4845a92f5215163a748b P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:02.416883 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 401be52e18ca4845a92f5215163a748b (table rebalancer_dynamic_table_new_name [id=6303650c8b074e7585874da5709d09fe]) successfully deleted
I20240502 01:59:02.417270 3477 ts_tablet_manager.cc:1875] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:02.419242 3477 ts_tablet_manager.cc:1888] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:02.419364 3477 log.cc:1196] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/401be52e18ca4845a92f5215163a748b
I20240502 01:59:02.419659 3477 ts_tablet_manager.cc:1909] T 401be52e18ca4845a92f5215163a748b P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:59:02.420370 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 401be52e18ca4845a92f5215163a748b (table rebalancer_dynamic_table_new_name [id=6303650c8b074e7585874da5709d09fe]) successfully deleted
I20240502 01:59:02.432126 3297 consensus_queue.cc:1035] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.436147 3297 consensus_queue.cc:1035] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.455771 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:02.455921 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:02.459304 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:02.459654 26774 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:02.460801 29059 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:02.461665 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.465072 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.492249 29196 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:02.494663 3494 ts_tablet_manager.cc:902] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:02.495334 3494 tablet_copy_client.cc:308] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:02.502238 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet 052566382186437e8e0245de9f60c30c from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:47995)
I20240502 01:59:02.502339 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet 052566382186437e8e0245de9f60c30c from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:47995: session id = 7c693d0933c64d9babc07f404df45cd1-052566382186437e8e0245de9f60c30c
I20240502 01:59:02.503181 26815 tablet_copy_source_session.cc:205] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:02.503986 3494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052566382186437e8e0245de9f60c30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.505954 3494 tablet_copy_client.cc:783] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:02.506119 3494 tablet_copy_client.cc:647] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:02.508704 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:02.518429 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:02.520684 29059 tablet_service.cc:1467] Processing CreateTablet for tablet d95acf4e375e419ea72b53a530cb9659 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]), partition=
I20240502 01:59:02.520900 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d95acf4e375e419ea72b53a530cb9659. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.520957 29468 tablet_service.cc:1467] Processing CreateTablet for tablet d95acf4e375e419ea72b53a530cb9659 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]), partition=
I20240502 01:59:02.521173 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d95acf4e375e419ea72b53a530cb9659. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.523877 29196 tablet_service.cc:1467] Processing CreateTablet for tablet d95acf4e375e419ea72b53a530cb9659 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]), partition=
I20240502 01:59:02.524096 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d95acf4e375e419ea72b53a530cb9659. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.524240 3500 tablet_bootstrap.cc:492] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:02.524883 3494 tablet_copy_client.cc:523] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:02.525665 3500 tablet_bootstrap.cc:654] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.526342 29329 tablet_service.cc:1467] Processing CreateTablet for tablet e96f6477ca7a497c82c01ef0b2d2747f (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.526607 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96f6477ca7a497c82c01ef0b2d2747f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.526791 29194 tablet_service.cc:1467] Processing CreateTablet for tablet 43129d24a99c45c7a2948fd113312bf7 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.526998 29194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43129d24a99c45c7a2948fd113312bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.527032 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 4b997cb0f369466eacadf8d56b678704 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.527235 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b997cb0f369466eacadf8d56b678704. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.528925 3501 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:02.528993 26774 tablet_service.cc:1467] Processing CreateTablet for tablet e96f6477ca7a497c82c01ef0b2d2747f (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.529253 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96f6477ca7a497c82c01ef0b2d2747f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.529312 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 4b997cb0f369466eacadf8d56b678704 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.529407 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 4b997cb0f369466eacadf8d56b678704 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.529543 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b997cb0f369466eacadf8d56b678704. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.529661 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b997cb0f369466eacadf8d56b678704. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.532229 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 43129d24a99c45c7a2948fd113312bf7 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.525967 29195 tablet_service.cc:1467] Processing CreateTablet for tablet e96f6477ca7a497c82c01ef0b2d2747f (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.532390 3501 tablet_bootstrap.cc:654] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.532449 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43129d24a99c45c7a2948fd113312bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.532449 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96f6477ca7a497c82c01ef0b2d2747f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.533262 3501 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:02.533356 3501 ts_tablet_manager.cc:1367] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:02.533835 3501 raft_consensus.cc:348] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.534018 3501 raft_consensus.cc:374] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.534132 3501 raft_consensus.cc:729] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.534327 3501 consensus_queue.cc:260] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.534674 3501 ts_tablet_manager.cc:1398] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:02.535284 3504 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:02.535634 3501 tablet_bootstrap.cc:492] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:02.535871 3502 tablet_bootstrap.cc:492] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:02.536571 3501 tablet_bootstrap.cc:654] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.537194 3504 tablet_bootstrap.cc:654] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.537927 3502 tablet_bootstrap.cc:654] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.538398 3501 tablet_bootstrap.cc:492] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:02.538482 3501 ts_tablet_manager.cc:1367] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:02.538867 3501 raft_consensus.cc:348] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.538966 3501 raft_consensus.cc:374] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.538997 3501 raft_consensus.cc:729] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.539103 3501 consensus_queue.cc:260] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.539273 3504 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:02.539342 3504 ts_tablet_manager.cc:1367] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:02.539345 3501 ts_tablet_manager.cc:1398] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:02.539779 3504 raft_consensus.cc:348] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.539904 3504 raft_consensus.cc:374] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.539937 3504 raft_consensus.cc:729] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.540046 3504 consensus_queue.cc:260] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.540375 3504 ts_tablet_manager.cc:1398] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:02.540988 3502 tablet_bootstrap.cc:492] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:02.541115 3502 ts_tablet_manager.cc:1367] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:02.541591 3502 raft_consensus.cc:348] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.541765 3502 raft_consensus.cc:374] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.541847 3502 raft_consensus.cc:729] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.542011 3502 consensus_queue.cc:260] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.542435 3502 ts_tablet_manager.cc:1398] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:02.542692 3502 tablet_bootstrap.cc:492] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:02.543684 3502 tablet_bootstrap.cc:654] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.544871 3502 tablet_bootstrap.cc:492] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:02.544977 3502 ts_tablet_manager.cc:1367] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:02.545714 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 43129d24a99c45c7a2948fd113312bf7 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:02.545956 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43129d24a99c45c7a2948fd113312bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:02.546571 3502 raft_consensus.cc:348] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.546777 3502 raft_consensus.cc:374] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.546849 3502 raft_consensus.cc:729] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.546969 3502 consensus_queue.cc:260] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.547329 3502 ts_tablet_manager.cc:1398] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:02.547547 3502 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:02.548303 3502 tablet_bootstrap.cc:654] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.550076 3502 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:02.550213 3502 ts_tablet_manager.cc:1367] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:59:02.550555 3494 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:02.550995 3502 raft_consensus.cc:348] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.551322 3502 raft_consensus.cc:374] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.551412 3502 raft_consensus.cc:729] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.551568 3502 consensus_queue.cc:260] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.551904 3502 ts_tablet_manager.cc:1398] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:02.553570 3499 tablet_bootstrap.cc:492] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:02.554453 3499 tablet_bootstrap.cc:654] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.554675 3505 tablet_bootstrap.cc:492] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:02.555596 3505 tablet_bootstrap.cc:654] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.561888 3499 tablet_bootstrap.cc:492] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:02.562053 3499 ts_tablet_manager.cc:1367] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:59:02.562503 3499 raft_consensus.cc:348] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.562659 3499 raft_consensus.cc:374] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.562726 3499 raft_consensus.cc:729] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.562894 3499 consensus_queue.cc:260] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.563211 3499 ts_tablet_manager.cc:1398] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:02.564486 3499 tablet_bootstrap.cc:492] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:02.565460 3499 tablet_bootstrap.cc:654] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.567548 3499 tablet_bootstrap.cc:492] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:02.567684 3499 ts_tablet_manager.cc:1367] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:02.568087 3499 raft_consensus.cc:348] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.568234 3499 raft_consensus.cc:374] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.568308 3499 raft_consensus.cc:729] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.568460 3499 consensus_queue.cc:260] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.568758 3499 ts_tablet_manager.cc:1398] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:02.570670 3505 tablet_bootstrap.cc:492] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:02.570801 3505 ts_tablet_manager.cc:1367] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 01:59:02.571918 3505 raft_consensus.cc:348] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.572075 3505 raft_consensus.cc:374] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.572140 3505 raft_consensus.cc:729] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.572324 3505 consensus_queue.cc:260] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.572698 3505 ts_tablet_manager.cc:1398] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:02.572907 3499 tablet_bootstrap.cc:492] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:02.574470 3499 tablet_bootstrap.cc:654] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.575758 3500 tablet_bootstrap.cc:492] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:02.575860 3500 ts_tablet_manager.cc:1367] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.002s
I20240502 01:59:02.576313 3500 raft_consensus.cc:348] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.576395 3500 raft_consensus.cc:374] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.576413 3500 raft_consensus.cc:729] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.576508 3500 consensus_queue.cc:260] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.576776 3500 ts_tablet_manager.cc:1398] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:02.577152 3500 tablet_bootstrap.cc:492] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:02.578069 3500 tablet_bootstrap.cc:654] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:02.580843 3500 tablet_bootstrap.cc:492] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:02.581959 3499 tablet_bootstrap.cc:492] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:02.582131 3499 ts_tablet_manager.cc:1367] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:59:02.582520 3499 raft_consensus.cc:348] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.582592 3499 raft_consensus.cc:374] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.582612 3499 raft_consensus.cc:729] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.582701 3499 consensus_queue.cc:260] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.583590 3499 ts_tablet_manager.cc:1398] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:02.583738 3500 ts_tablet_manager.cc:1367] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 01:59:02.584144 3500 raft_consensus.cc:348] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.584228 3500 raft_consensus.cc:374] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:02.584247 3500 raft_consensus.cc:729] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.584328 3500 consensus_queue.cc:260] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.584504 3500 ts_tablet_manager.cc:1398] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:02.589635 3252 raft_consensus.cc:482] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:02.589743 3252 raft_consensus.cc:504] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.590257 3252 leader_election.cc:288] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:02.590483 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96f6477ca7a497c82c01ef0b2d2747f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:02.590608 29213 raft_consensus.cc:2454] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:02.590641 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96f6477ca7a497c82c01ef0b2d2747f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:02.590757 29350 raft_consensus.cc:2454] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:02.591135 26727 leader_election.cc:302] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:02.591432 3252 raft_consensus.cc:2781] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:02.591511 3252 raft_consensus.cc:482] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:02.591554 3252 raft_consensus.cc:3037] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.592490 3252 raft_consensus.cc:504] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.592810 3252 leader_election.cc:288] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:02.593320 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96f6477ca7a497c82c01ef0b2d2747f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:02.593434 29213 raft_consensus.cc:3037] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.593643 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96f6477ca7a497c82c01ef0b2d2747f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:02.593760 29350 raft_consensus.cc:3037] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.594334 29213 raft_consensus.cc:2454] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:02.594763 29350 raft_consensus.cc:2454] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:02.595033 26729 leader_election.cc:302] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:02.595201 3252 raft_consensus.cc:2781] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:02.595291 3252 raft_consensus.cc:686] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:02.595407 3252 consensus_queue.cc:237] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:02.595788 3494 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:02.596048 3494 tablet_bootstrap.cc:492] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:59:02.596216 3494 ts_tablet_manager.cc:1367] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.046s user 0.019s sys 0.004s
I20240502 01:59:02.596598 3494 raft_consensus.cc:348] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:02.596714 3494 raft_consensus.cc:729] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:02.596855 3494 consensus_queue.cc:260] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:02.597211 3494 ts_tablet_manager.cc:1398] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240502 01:59:02.597841 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-052566382186437e8e0245de9f60c30c received from {username='slave'} at 127.19.43.200:47995
I20240502 01:59:02.597916 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-052566382186437e8e0245de9f60c30c on tablet 052566382186437e8e0245de9f60c30c with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:02.598443 26509 catalog_manager.cc:5484] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:02.604480 29350 consensus_queue.cc:237] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:02.605818 26927 raft_consensus.cc:1261] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:02.605969 29213 raft_consensus.cc:1261] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:02.606638 3421 consensus_queue.cc:1035] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:02.606964 3421 consensus_queue.cc:1035] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:02.608361 3421 raft_consensus.cc:2932] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:02.609305 26927 raft_consensus.cc:2932] T 01764f37959e4c6799e3a83f0d4ed2e1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:02.609314 29213 raft_consensus.cc:2932] T 01764f37959e4c6799e3a83f0d4ed2e1 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:02.612329 26513 catalog_manager.cc:5484] T 01764f37959e4c6799e3a83f0d4ed2e1 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 2 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:02.615093 3422 raft_consensus.cc:482] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:02.615195 3422 raft_consensus.cc:504] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.615569 3422 leader_election.cc:288] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:02.615936 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 01764f37959e4c6799e3a83f0d4ed2e1 with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:02.616189 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b997cb0f369466eacadf8d56b678704" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:02.616309 29489 raft_consensus.cc:2454] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:02.616912 29016 leader_election.cc:302] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:02.617062 3422 raft_consensus.cc:2781] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:02.617115 3422 raft_consensus.cc:482] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:02.617148 3422 raft_consensus.cc:3037] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.618284 3422 raft_consensus.cc:504] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.618515 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b997cb0f369466eacadf8d56b678704" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:02.618626 26927 raft_consensus.cc:2454] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:02.618619 3422 leader_election.cc:288] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:02.620402 3539 tablet_replica.cc:330] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:02.620689 3539 raft_consensus.cc:2229] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:02.620889 3539 raft_consensus.cc:2258] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:02.621286 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b997cb0f369466eacadf8d56b678704" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:02.621384 29489 raft_consensus.cc:3037] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.621419 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b997cb0f369466eacadf8d56b678704" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:02.621505 26927 raft_consensus.cc:3037] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.622308 29489 raft_consensus.cc:2454] T 4b997cb0f369466eacadf8d56b678704 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:02.622345 26927 raft_consensus.cc:2454] T 4b997cb0f369466eacadf8d56b678704 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:02.622588 3539 ts_tablet_manager.cc:1875] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:02.622608 29016 leader_election.cc:302] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:02.622763 3422 raft_consensus.cc:2781] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:02.622846 3422 raft_consensus.cc:686] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:02.622952 3422 consensus_queue.cc:237] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:02.625706 26506 catalog_manager.cc:5484] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:02.628713 3539 ts_tablet_manager.cc:1888] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:02.628779 3539 log.cc:1196] T 01764f37959e4c6799e3a83f0d4ed2e1 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/01764f37959e4c6799e3a83f0d4ed2e1
I20240502 01:59:02.629251 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 01764f37959e4c6799e3a83f0d4ed2e1 (table rebalancer_extra_table_8 [id=3c38ff9844b8486f8c4cb82921867437]) successfully deleted
I20240502 01:59:02.631842 2712 raft_consensus.cc:482] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:02.631932 2712 raft_consensus.cc:504] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.632309 2712 leader_election.cc:288] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:02.632623 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43129d24a99c45c7a2948fd113312bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:02.632623 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43129d24a99c45c7a2948fd113312bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:02.632733 29213 raft_consensus.cc:2454] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:02.632733 29350 raft_consensus.cc:2454] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:02.632980 29013 leader_election.cc:302] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:02.633128 2712 raft_consensus.cc:2781] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:02.633188 2712 raft_consensus.cc:482] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:02.633215 2712 raft_consensus.cc:3037] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.634157 2712 raft_consensus.cc:504] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.634457 2712 leader_election.cc:288] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:02.634831 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43129d24a99c45c7a2948fd113312bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:02.634935 29213 raft_consensus.cc:3037] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.634831 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43129d24a99c45c7a2948fd113312bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:02.635294 29350 raft_consensus.cc:3037] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.635811 29213 raft_consensus.cc:2454] T 43129d24a99c45c7a2948fd113312bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:02.636080 29015 leader_election.cc:302] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:02.636242 2712 raft_consensus.cc:2781] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:02.636276 29350 raft_consensus.cc:2454] T 43129d24a99c45c7a2948fd113312bf7 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:02.636314 2712 raft_consensus.cc:686] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:02.636422 2712 consensus_queue.cc:237] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.638454 26506 catalog_manager.cc:5484] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:02.703995 1595 raft_consensus.cc:482] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:02.704109 1595 raft_consensus.cc:504] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.704591 1595 leader_election.cc:288] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:02.704982 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95acf4e375e419ea72b53a530cb9659" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:02.705118 29079 raft_consensus.cc:2454] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:02.705435 29419 leader_election.cc:302] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:02.705709 1595 raft_consensus.cc:2781] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:02.705772 1595 raft_consensus.cc:482] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:02.705819 1595 raft_consensus.cc:3037] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.705853 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95acf4e375e419ea72b53a530cb9659" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:02.705999 29213 raft_consensus.cc:2454] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:02.706959 1595 raft_consensus.cc:504] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.707336 1595 leader_election.cc:288] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:02.707684 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95acf4e375e419ea72b53a530cb9659" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:02.707799 29213 raft_consensus.cc:3037] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.708226 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95acf4e375e419ea72b53a530cb9659" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:02.708328 29079 raft_consensus.cc:3037] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:02.708719 29213 raft_consensus.cc:2454] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:02.709095 29421 leader_election.cc:302] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:02.709249 2101 raft_consensus.cc:2781] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:02.709268 29079 raft_consensus.cc:2454] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:02.709383 2101 raft_consensus.cc:686] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:02.709519 2101 consensus_queue.cc:237] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:02.714344 26511 catalog_manager.cc:5484] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:02.740144 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:02.748391 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.750324 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:02.751173 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:02.755224 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:02.765030 29468 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:02.774091 29059 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:02.802212 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:02.835117 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:02.839532 29213 raft_consensus.cc:1261] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:02.839605 29080 raft_consensus.cc:1261] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:02.839996 3584 consensus_queue.cc:1035] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.840466 2101 consensus_queue.cc:1035] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:02.842749 26508 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]). Expected version 1 got 0
I20240502 01:59:02.844767 26514 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]). Expected version 1 got 0
I20240502 01:59:02.844995 3610 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:02.845296 3610 tablet.cc:1620] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:02.845669 3609 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:02.845991 3609 tablet.cc:1620] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:02.846493 3611 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:02.847302 26514 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]). Expected version 1 got 0
I20240502 01:59:02.848769 26508 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d]). Expected version 1 got 0
I20240502 01:59:02.849120 3611 tablet.cc:1620] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:02.853418 26508 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d] alter complete (version 1)
I20240502 01:59:02.859383 26794 consensus_queue.cc:237] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:02.860469 29489 raft_consensus.cc:1261] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:02.860471 26659 raft_consensus.cc:1261] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:02.860869 3252 consensus_queue.cc:1035] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:02.861057 3489 consensus_queue.cc:1035] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:02.862816 3489 raft_consensus.cc:2932] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:02.863164 26659 raft_consensus.cc:2932] T 49ac3a00934944bdb5c9e440045532a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:02.863482 29489 raft_consensus.cc:2932] T 49ac3a00934944bdb5c9e440045532a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:02.866008 26514 catalog_manager.cc:5484] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:02.870530 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 49ac3a00934944bdb5c9e440045532a3 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:02.871218 3626 tablet_replica.cc:330] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:02.871405 3626 raft_consensus.cc:2229] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:02.871544 3626 raft_consensus.cc:2258] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:02.871958 3626 ts_tablet_manager.cc:1875] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:02.873842 3626 ts_tablet_manager.cc:1888] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:59:02.873942 3626 log.cc:1196] T 49ac3a00934944bdb5c9e440045532a3 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/49ac3a00934944bdb5c9e440045532a3
I20240502 01:59:02.875630 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 49ac3a00934944bdb5c9e440045532a3 (table rebalance_test_table_9 [id=911a1465522146b09f03a60b31759670]) successfully deleted
I20240502 01:59:02.881664 29080 raft_consensus.cc:1203] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:59:02.889573 29489 raft_consensus.cc:1203] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:59:02.958863 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:02.965318 3609 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:02.965617 3609 tablet.cc:1620] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:02.966398 3611 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:02.966678 3611 tablet.cc:1620] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:02.975267 3610 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:02.975584 3610 tablet.cc:1620] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:02.977603 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:02.978789 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=d8d1e4b520bf47328be089ca47c9070d] alter complete (version 2)
I20240502 01:59:02.989954 29468 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:02.991987 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:02.993611 27044 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:03.001516 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:03.007068 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.014905 3422 consensus_queue.cc:1035] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.019714 29059 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:03.022087 2712 consensus_queue.cc:1035] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.030768 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.066618 3489 consensus_queue.cc:1035] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.069582 3489 consensus_queue.cc:1035] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.074153 26794 consensus_queue.cc:237] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:03.075572 26927 raft_consensus.cc:1261] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:03.075592 29489 raft_consensus.cc:1261] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:03.076031 3489 consensus_queue.cc:1035] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:03.076213 3252 consensus_queue.cc:1035] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:03.077804 3489 raft_consensus.cc:2932] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.078490 29487 raft_consensus.cc:2932] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.078498 26927 raft_consensus.cc:2932] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.081287 26508 catalog_manager.cc:5484] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.084257 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:03.084491 3626 tablet_replica.cc:330] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:03.084645 3626 raft_consensus.cc:2229] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:03.084795 3626 raft_consensus.cc:2258] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:03.085317 3626 ts_tablet_manager.cc:1875] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:03.087325 3626 ts_tablet_manager.cc:1888] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:59:03.087390 3626 log.cc:1196] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/7b2aaf8eaa844c0390909ccfc0b4bb09
I20240502 01:59:03.087813 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 7b2aaf8eaa844c0390909ccfc0b4bb09 (table rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]) successfully deleted
I20240502 01:59:03.088155 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:03.093577 3609 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:03.094069 3610 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:03.094588 3611 tablet.cc:1722] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:03.099671 2712 consensus_queue.cc:1035] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.099867 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=d8d1e4b520bf47328be089ca47c9070d] alter complete (version 3)
I20240502 01:59:03.106149 2712 consensus_queue.cc:1035] T 43129d24a99c45c7a2948fd113312bf7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.121544 26794 consensus_queue.cc:237] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.122823 26927 raft_consensus.cc:1261] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:03.122829 26659 raft_consensus.cc:1261] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:03.123235 3620 consensus_queue.cc:1035] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:59:03.123570 26729 consensus_peers.cc:485] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dcf3a090ea284d12979b161f59081ff7. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:03.123953 3620 consensus_queue.cc:1035] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:03.125564 3489 raft_consensus.cc:2932] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:03.126010 26927 raft_consensus.cc:2932] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:03.127894 26661 raft_consensus.cc:2932] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:03.128698 26510 catalog_manager.cc:5484] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:03.158382 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:03.164145 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 2d96c05a484343ad86bbdefd2a59044f (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.164443 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d96c05a484343ad86bbdefd2a59044f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.164570 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 2d96c05a484343ad86bbdefd2a59044f (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.164835 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d96c05a484343ad86bbdefd2a59044f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.166850 3747 tablet_bootstrap.cc:492] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:03.167112 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 2d96c05a484343ad86bbdefd2a59044f (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.167387 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d96c05a484343ad86bbdefd2a59044f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.167866 3747 tablet_bootstrap.cc:654] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.169471 3750 tablet_bootstrap.cc:492] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:03.169704 26908 tablet_service.cc:1467] Processing CreateTablet for tablet b546a99821624f0588a4c0f59a5ec5a3 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.169958 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b546a99821624f0588a4c0f59a5ec5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.169929 27044 tablet_service.cc:1467] Processing CreateTablet for tablet b546a99821624f0588a4c0f59a5ec5a3 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.169704 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 56412781175b453e8dd31dd87e3c6dc8 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.170215 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b546a99821624f0588a4c0f59a5ec5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.170349 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56412781175b453e8dd31dd87e3c6dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.172216 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 56412781175b453e8dd31dd87e3c6dc8 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.172459 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56412781175b453e8dd31dd87e3c6dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.172660 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 56412781175b453e8dd31dd87e3c6dc8 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.172847 3748 tablet_bootstrap.cc:492] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:03.172881 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56412781175b453e8dd31dd87e3c6dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.173658 3748 tablet_bootstrap.cc:654] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.174250 3751 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:03.174997 26640 tablet_service.cc:1467] Processing CreateTablet for tablet b546a99821624f0588a4c0f59a5ec5a3 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.175256 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b546a99821624f0588a4c0f59a5ec5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.176390 3751 tablet_bootstrap.cc:654] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.178061 3751 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:03.178170 3751 ts_tablet_manager.cc:1367] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:03.178361 3750 tablet_bootstrap.cc:654] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.178614 3751 raft_consensus.cc:348] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.178743 3751 raft_consensus.cc:374] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.178817 3751 raft_consensus.cc:729] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.178954 3751 consensus_queue.cc:260] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.179041 3756 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:03.179351 3751 ts_tablet_manager.cc:1398] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.179585 3751 tablet_bootstrap.cc:492] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:03.179766 3753 tablet_bootstrap.cc:492] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:03.180078 3750 tablet_bootstrap.cc:492] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:03.180243 3750 ts_tablet_manager.cc:1367] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:03.180474 3751 tablet_bootstrap.cc:654] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.180691 3753 tablet_bootstrap.cc:654] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.180747 3750 raft_consensus.cc:348] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.180899 3750 raft_consensus.cc:374] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.181241 3750 raft_consensus.cc:729] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.181454 3750 consensus_queue.cc:260] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.181562 3751 tablet_bootstrap.cc:492] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:03.181631 3751 ts_tablet_manager.cc:1367] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:03.181905 3750 ts_tablet_manager.cc:1398] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:03.182052 3751 raft_consensus.cc:348] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.182142 3751 raft_consensus.cc:374] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.182170 3751 raft_consensus.cc:729] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.182268 3751 consensus_queue.cc:260] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.182513 3751 ts_tablet_manager.cc:1398] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.182559 3752 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:03.183094 3756 tablet_bootstrap.cc:654] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.183521 3752 tablet_bootstrap.cc:654] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.184922 3756 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:03.185006 3756 ts_tablet_manager.cc:1367] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:03.185451 3756 raft_consensus.cc:348] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.185529 3752 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:03.185541 3756 raft_consensus.cc:374] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.185570 3756 raft_consensus.cc:729] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.185601 3752 ts_tablet_manager.cc:1367] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:03.185678 3756 consensus_queue.cc:260] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.185983 3756 ts_tablet_manager.cc:1398] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.186014 3752 raft_consensus.cc:348] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.186133 3752 raft_consensus.cc:374] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.186162 3752 raft_consensus.cc:729] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.186313 3752 consensus_queue.cc:260] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.186370 3747 tablet_bootstrap.cc:492] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:03.186453 3747 ts_tablet_manager.cc:1367] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 01:59:03.186569 3752 ts_tablet_manager.cc:1398] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:03.186904 3747 raft_consensus.cc:348] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.186993 3753 tablet_bootstrap.cc:492] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:03.187011 3747 raft_consensus.cc:374] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.187043 3747 raft_consensus.cc:729] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.187053 3753 ts_tablet_manager.cc:1367] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:03.187181 3747 consensus_queue.cc:260] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.187417 3747 ts_tablet_manager.cc:1398] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.187440 3753 raft_consensus.cc:348] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.187517 3753 raft_consensus.cc:374] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.187536 3753 raft_consensus.cc:729] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.187626 3753 consensus_queue.cc:260] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.187954 3753 ts_tablet_manager.cc:1398] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.189033 3752 tablet_bootstrap.cc:492] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:03.189893 3752 tablet_bootstrap.cc:654] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.191488 3752 tablet_bootstrap.cc:492] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:03.191563 3752 ts_tablet_manager.cc:1367] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:59:03.191944 3752 raft_consensus.cc:348] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.192049 3752 raft_consensus.cc:374] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.192071 3752 raft_consensus.cc:729] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.192164 3752 consensus_queue.cc:260] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.192420 3752 ts_tablet_manager.cc:1398] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:03.195155 3748 tablet_bootstrap.cc:492] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:03.195258 3748 ts_tablet_manager.cc:1367] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 01:59:03.195734 3748 raft_consensus.cc:348] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.195858 3748 raft_consensus.cc:374] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.195896 3748 raft_consensus.cc:729] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.196014 3748 consensus_queue.cc:260] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.196266 3748 ts_tablet_manager.cc:1398] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.202505 26515 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:03.202616 26515 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:03.204960 26515 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet d95acf4e375e419ea72b53a530cb9659
I20240502 01:59:03.205665 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet d95acf4e375e419ea72b53a530cb9659 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:03 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:03.205685 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet d95acf4e375e419ea72b53a530cb9659 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:03 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:03.206735 3789 tablet_replica.cc:330] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:03.206789 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet d95acf4e375e419ea72b53a530cb9659 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:03 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:03.206960 3789 raft_consensus.cc:2229] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:03.207152 3789 raft_consensus.cc:2258] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:03.207823 3789 ts_tablet_manager.cc:1875] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:03.212075 3789 ts_tablet_manager.cc:1888] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:03.212198 3789 log.cc:1196] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/d95acf4e375e419ea72b53a530cb9659
I20240502 01:59:03.212535 3789 ts_tablet_manager.cc:1909] T d95acf4e375e419ea72b53a530cb9659 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:03.213106 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table_new_name [id=d8d1e4b520bf47328be089ca47c9070d]) successfully deleted
I20240502 01:59:03.214138 3372 raft_consensus.cc:482] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:03.214242 3372 raft_consensus.cc:504] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.214494 3790 tablet_replica.cc:330] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:03.214587 3372 leader_election.cc:288] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:03.214764 3790 raft_consensus.cc:2229] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:03.214980 3790 raft_consensus.cc:2258] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:03.215067 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56412781175b453e8dd31dd87e3c6dc8" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:03.215176 26794 raft_consensus.cc:2454] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:03.215178 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56412781175b453e8dd31dd87e3c6dc8" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:03.215305 26927 raft_consensus.cc:2454] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:03.215524 3790 ts_tablet_manager.cc:1875] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:03.215605 26997 leader_election.cc:302] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:03.215746 2008 raft_consensus.cc:482] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:03.215803 2008 raft_consensus.cc:504] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.216154 2008 leader_election.cc:288] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:03.216228 2008 raft_consensus.cc:2781] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:03.216259 2008 raft_consensus.cc:482] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:03.216305 2008 raft_consensus.cc:3037] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.217231 2008 raft_consensus.cc:504] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.217527 2008 leader_election.cc:288] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:03.219372 3788 tablet_replica.cc:330] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:03.220372 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56412781175b453e8dd31dd87e3c6dc8" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:03.220485 26930 raft_consensus.cc:3037] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.221429 26930 raft_consensus.cc:2454] T 56412781175b453e8dd31dd87e3c6dc8 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:03.221509 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:03.221618 26661 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:03.222448 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56412781175b453e8dd31dd87e3c6dc8" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:03.222545 26794 raft_consensus.cc:3037] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.222553 3788 raft_consensus.cc:2229] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:03.222657 3788 raft_consensus.cc:2258] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:03.223162 3790 ts_tablet_manager.cc:1888] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:03.223223 3790 log.cc:1196] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/d95acf4e375e419ea72b53a530cb9659
I20240502 01:59:03.223255 3788 ts_tablet_manager.cc:1875] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:03.223459 3790 ts_tablet_manager.cc:1909] T d95acf4e375e419ea72b53a530cb9659 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:59:03.223467 26794 raft_consensus.cc:2454] T 56412781175b453e8dd31dd87e3c6dc8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:03.224223 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:03.224346 26927 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:03.224545 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table_new_name [id=d8d1e4b520bf47328be089ca47c9070d]) successfully deleted
I20240502 01:59:03.224567 26997 leader_election.cc:302] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:03.224704 26997 leader_election.cc:302] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:03.225071 3791 raft_consensus.cc:2781] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:03.225077 3372 raft_consensus.cc:2781] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:03.225229 3372 raft_consensus.cc:686] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:03.225250 3791 raft_consensus.cc:482] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:03.225374 3791 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.225543 3788 ts_tablet_manager.cc:1888] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:03.225359 3372 consensus_queue.cc:237] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.228246 26510 catalog_manager.cc:5484] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:03.228705 3791 raft_consensus.cc:504] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.229032 3791 leader_election.cc:288] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:03.229346 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:03.229457 26927 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.229434 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:03.229529 26661 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.229920 3788 log.cc:1196] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/d95acf4e375e419ea72b53a530cb9659
I20240502 01:59:03.230274 3788 ts_tablet_manager.cc:1909] T d95acf4e375e419ea72b53a530cb9659 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:03.230343 26927 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:03.230464 26661 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:03.230932 26998 leader_election.cc:302] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:03.231009 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet d95acf4e375e419ea72b53a530cb9659 (table rebalancer_dynamic_table_new_name [id=d8d1e4b520bf47328be089ca47c9070d]) successfully deleted
I20240502 01:59:03.231101 3791 raft_consensus.cc:2781] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:03.231379 3791 raft_consensus.cc:686] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:03.231508 3791 consensus_queue.cc:237] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.235229 26512 catalog_manager.cc:5484] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:03.281399 2712 raft_consensus.cc:482] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:03.281512 2712 raft_consensus.cc:504] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.281894 2712 leader_election.cc:288] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:03.282195 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d96c05a484343ad86bbdefd2a59044f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:03.282210 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d96c05a484343ad86bbdefd2a59044f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:03.282334 29216 raft_consensus.cc:2454] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:03.282334 29349 raft_consensus.cc:2454] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:03.282581 29013 leader_election.cc:302] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:03.282739 2712 raft_consensus.cc:2781] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:03.282810 2712 raft_consensus.cc:482] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:03.282858 2712 raft_consensus.cc:3037] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.283600 2712 raft_consensus.cc:504] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.283877 2712 leader_election.cc:288] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:03.284147 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d96c05a484343ad86bbdefd2a59044f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:03.284193 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d96c05a484343ad86bbdefd2a59044f" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:03.284246 29216 raft_consensus.cc:3037] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.284313 29349 raft_consensus.cc:3037] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.285104 29216 raft_consensus.cc:2454] T 2d96c05a484343ad86bbdefd2a59044f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:03.285207 29349 raft_consensus.cc:2454] T 2d96c05a484343ad86bbdefd2a59044f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:03.285373 29015 leader_election.cc:302] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:03.285529 2712 raft_consensus.cc:2781] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:03.285702 2712 raft_consensus.cc:686] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:03.285835 2712 consensus_queue.cc:237] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:03.287878 26506 catalog_manager.cc:5484] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:03.306421 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:03.307395 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:03.321887 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 02aeb9bbfe404f8587bcdc97b47eb006 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492]), partition=
I20240502 01:59:03.322105 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 02aeb9bbfe404f8587bcdc97b47eb006 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492]), partition=
I20240502 01:59:03.322144 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02aeb9bbfe404f8587bcdc97b47eb006. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.322151 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 02aeb9bbfe404f8587bcdc97b47eb006 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492]), partition=
I20240502 01:59:03.322328 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02aeb9bbfe404f8587bcdc97b47eb006. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.322355 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02aeb9bbfe404f8587bcdc97b47eb006. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.324730 3752 tablet_bootstrap.cc:492] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:03.325626 3752 tablet_bootstrap.cc:654] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.326117 3747 tablet_bootstrap.cc:492] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:03.326138 3748 tablet_bootstrap.cc:492] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:03.326938 3748 tablet_bootstrap.cc:654] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.327051 3747 tablet_bootstrap.cc:654] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.330529 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.330574 3748 tablet_bootstrap.cc:492] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:03.330641 3748 ts_tablet_manager.cc:1367] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:03.331060 3748 raft_consensus.cc:348] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.331173 3748 raft_consensus.cc:374] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.331209 3748 raft_consensus.cc:729] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.331333 3748 consensus_queue.cc:260] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.331614 3748 ts_tablet_manager.cc:1398] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.331967 3747 tablet_bootstrap.cc:492] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:03.332039 3747 ts_tablet_manager.cc:1367] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:03.332435 3747 raft_consensus.cc:348] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.332526 3747 raft_consensus.cc:374] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.332554 3747 raft_consensus.cc:729] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.332655 3747 consensus_queue.cc:260] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.332916 3747 ts_tablet_manager.cc:1398] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.333248 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:03.334687 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:03.334774 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.338135 3752 tablet_bootstrap.cc:492] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:03.338217 3752 ts_tablet_manager.cc:1367] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:03.338618 3752 raft_consensus.cc:348] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.338713 3752 raft_consensus.cc:374] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.338744 3752 raft_consensus.cc:729] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.338865 3752 consensus_queue.cc:260] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.339124 3752 ts_tablet_manager.cc:1398] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:03.344489 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:03.345108 29059 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:03.348727 27043 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:03.538739 2712 raft_consensus.cc:482] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:03.538836 2712 raft_consensus.cc:504] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.539228 2712 leader_election.cc:288] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:03.539469 3791 raft_consensus.cc:482] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:03.539485 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:03.539522 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:03.539584 3791 raft_consensus.cc:504] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.539631 29216 raft_consensus.cc:2454] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:03.539878 27063 raft_consensus.cc:2454] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:03.539891 29015 leader_election.cc:302] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:03.540042 2712 raft_consensus.cc:2781] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:03.540097 2712 raft_consensus.cc:482] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:03.540093 3791 leader_election.cc:288] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:03.540133 2712 raft_consensus.cc:3037] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.540555 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:03.540557 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:03.540658 29216 raft_consensus.cc:2454] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:03.540892 26997 leader_election.cc:302] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:03.541043 3791 raft_consensus.cc:2781] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:03.541091 3791 raft_consensus.cc:482] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:03.541136 3791 raft_consensus.cc:3037] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.541262 2712 raft_consensus.cc:504] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.541429 29080 raft_consensus.cc:2379] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68e5b1eeb04448fea818a14991eab87c in current term 1: Already voted for candidate f3d18a019013487ab801599cfb80e4b4 in this term.
I20240502 01:59:03.541978 2712 leader_election.cc:288] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:03.542063 3791 raft_consensus.cc:504] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.542127 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:03.542200 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:03.542286 27063 raft_consensus.cc:2379] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3d18a019013487ab801599cfb80e4b4 in current term 1: Already voted for candidate 68e5b1eeb04448fea818a14991eab87c in this term.
I20240502 01:59:03.542275 29216 raft_consensus.cc:3037] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.542382 3791 leader_election.cc:288] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:03.542719 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:03.542721 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02aeb9bbfe404f8587bcdc97b47eb006" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:03.542845 29080 raft_consensus.cc:2379] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68e5b1eeb04448fea818a14991eab87c in current term 1: Already voted for candidate f3d18a019013487ab801599cfb80e4b4 in this term.
I20240502 01:59:03.543118 26995 leader_election.cc:302] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:03.543200 29216 raft_consensus.cc:2454] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:03.543440 29015 leader_election.cc:302] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters: 68e5b1eeb04448fea818a14991eab87c
I20240502 01:59:03.543502 3791 raft_consensus.cc:2726] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 01:59:03.543622 2712 raft_consensus.cc:2781] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:03.543785 2712 raft_consensus.cc:686] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:03.543915 2712 consensus_queue.cc:237] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.545897 26510 catalog_manager.cc:5484] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:03.569622 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:03.570621 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:03.579192 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.581266 27043 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:03.581509 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.582222 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:03.585749 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:03.591473 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:03.627306 3863 ts_tablet_manager.cc:902] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:03.627861 3863 tablet_copy_client.cc:308] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:03.629200 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet dcf3a090ea284d12979b161f59081ff7 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:56195)
I20240502 01:59:03.629303 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet dcf3a090ea284d12979b161f59081ff7 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:56195: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-dcf3a090ea284d12979b161f59081ff7
I20240502 01:59:03.630265 26815 tablet_copy_source_session.cc:205] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:03.631217 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcf3a090ea284d12979b161f59081ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.633523 3863 tablet_copy_client.cc:783] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:03.633677 3863 tablet_copy_client.cc:647] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:03.634732 3863 tablet_copy_client.cc:523] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:03.635900 3863 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:03.656428 3863 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:03.656627 3863 tablet_bootstrap.cc:492] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:59:03.656759 3863 ts_tablet_manager.cc:1367] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20240502 01:59:03.657150 3863 raft_consensus.cc:348] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.657276 3863 raft_consensus.cc:729] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.657402 3863 consensus_queue.cc:260] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.657685 3863 ts_tablet_manager.cc:1398] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 01:59:03.658190 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-dcf3a090ea284d12979b161f59081ff7 received from {username='slave'} at 127.19.43.198:56195
I20240502 01:59:03.658258 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-dcf3a090ea284d12979b161f59081ff7 on tablet dcf3a090ea284d12979b161f59081ff7 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:59:03.676406 26661 consensus_queue.cc:237] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:03.677487 29080 raft_consensus.cc:1261] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:03.677585 26793 raft_consensus.cc:1261] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:03.678058 3676 consensus_queue.cc:1035] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:03.678258 3885 consensus_queue.cc:1035] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:03.679333 3791 consensus_queue.cc:1035] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.679958 3676 raft_consensus.cc:2932] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:03.680429 29080 raft_consensus.cc:2932] T 69df2cdb31d640ff9f43bb395f2f26fe P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:03.680450 26794 raft_consensus.cc:2932] T 69df2cdb31d640ff9f43bb395f2f26fe P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:03.683063 26510 catalog_manager.cc:5484] T 69df2cdb31d640ff9f43bb395f2f26fe P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:03.686132 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 69df2cdb31d640ff9f43bb395f2f26fe with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:03.686476 3889 tablet_replica.cc:330] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:03.686666 3889 raft_consensus.cc:2229] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:03.686873 3889 raft_consensus.cc:2258] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:03.688349 3889 ts_tablet_manager.cc:1875] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:03.689137 3791 consensus_queue.cc:1035] T 56412781175b453e8dd31dd87e3c6dc8 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.691483 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:03.692145 3889 ts_tablet_manager.cc:1888] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:59:03.692242 3889 log.cc:1196] T 69df2cdb31d640ff9f43bb395f2f26fe P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/69df2cdb31d640ff9f43bb395f2f26fe
I20240502 01:59:03.692735 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 69df2cdb31d640ff9f43bb395f2f26fe (table rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]) successfully deleted
I20240502 01:59:03.694903 27063 raft_consensus.cc:1261] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:03.695035 29216 raft_consensus.cc:1261] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:03.695330 2712 consensus_queue.cc:1035] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.695456 3794 consensus_queue.cc:1035] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.698464 26506 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 02aeb9bbfe404f8587bcdc97b47eb006 (table rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492]). Expected version 1 got 0
I20240502 01:59:03.699112 26508 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 02aeb9bbfe404f8587bcdc97b47eb006 (table rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492]). Expected version 1 got 0
I20240502 01:59:03.699358 3909 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:03.700181 3909 tablet.cc:1620] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:03.700870 3911 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:03.701151 3911 tablet.cc:1620] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:03.702154 3912 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:03.702417 3912 tablet.cc:1620] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:03.702853 26506 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet 02aeb9bbfe404f8587bcdc97b47eb006 (table rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492]). Expected version 1 got 0
I20240502 01:59:03.706383 26506 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492] alter complete (version 1)
I20240502 01:59:03.719419 3372 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.722084 3791 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.763221 3794 consensus_queue.cc:1035] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.768455 3422 consensus_queue.cc:1035] T 2d96c05a484343ad86bbdefd2a59044f P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:03.799000 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:03.804621 27043 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:03.810875 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:03.813614 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:03.813892 26908 tablet_service.cc:1467] Processing CreateTablet for tablet ab7a9b176213400f990c4d5df0d7567f (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.813935 29329 tablet_service.cc:1467] Processing CreateTablet for tablet ab7a9b176213400f990c4d5df0d7567f (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.814198 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab7a9b176213400f990c4d5df0d7567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.814198 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab7a9b176213400f990c4d5df0d7567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.817036 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:03.817085 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 213c37d37f1c4a3c8fbc48deafa9edac (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.817327 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213c37d37f1c4a3c8fbc48deafa9edac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.817431 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 213c37d37f1c4a3c8fbc48deafa9edac (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.817662 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213c37d37f1c4a3c8fbc48deafa9edac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.819793 3747 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:03.820713 3747 tablet_bootstrap.cc:654] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.821678 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 39dd23a3674d4beea982741db70f16f8 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.821895 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dd23a3674d4beea982741db70f16f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.822088 29329 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:03.822386 26640 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.823004 3963 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:03.824680 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 39dd23a3674d4beea982741db70f16f8 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.824908 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dd23a3674d4beea982741db70f16f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.827117 27043 tablet_service.cc:1467] Processing CreateTablet for tablet ab7a9b176213400f990c4d5df0d7567f (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.827387 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab7a9b176213400f990c4d5df0d7567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.828006 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 213c37d37f1c4a3c8fbc48deafa9edac (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.828224 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213c37d37f1c4a3c8fbc48deafa9edac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.829396 3752 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:03.830231 3752 tablet_bootstrap.cc:654] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.830411 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 39dd23a3674d4beea982741db70f16f8 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:03.830627 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dd23a3674d4beea982741db70f16f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:03.834101 3752 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:03.834192 3752 ts_tablet_manager.cc:1367] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:03.834640 3752 raft_consensus.cc:348] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.834740 3752 raft_consensus.cc:374] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.834770 3752 raft_consensus.cc:729] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.834878 3752 consensus_queue.cc:260] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.835163 3752 ts_tablet_manager.cc:1398] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.835327 3752 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:03.836165 3752 tablet_bootstrap.cc:654] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.836350 3912 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:03.836578 3912 tablet.cc:1620] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:03.836777 3963 tablet_bootstrap.cc:654] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.837445 3909 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:03.837706 3909 tablet.cc:1620] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:03.837752 3966 tablet_bootstrap.cc:492] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:03.841825 3911 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:03.842087 3911 tablet.cc:1620] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:03.842422 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:03.845562 3966 tablet_bootstrap.cc:654] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.845799 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:03.849474 26506 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=b567d66ec9b241a8b2f8568d554d9492] alter complete (version 2)
I20240502 01:59:03.855687 3747 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:03.855774 3747 ts_tablet_manager.cc:1367] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240502 01:59:03.856153 3747 raft_consensus.cc:348] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.856246 3747 raft_consensus.cc:374] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.856273 3747 raft_consensus.cc:729] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.856375 3747 consensus_queue.cc:260] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.856626 3747 ts_tablet_manager.cc:1398] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.861240 3961 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:03.862912 3961 tablet_bootstrap.cc:654] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.864033 3963 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:03.864997 3963 ts_tablet_manager.cc:1367] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20240502 01:59:03.865947 3963 raft_consensus.cc:348] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.866159 3963 raft_consensus.cc:374] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.866240 3963 raft_consensus.cc:729] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.866500 3963 consensus_queue.cc:260] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.867703 3963 ts_tablet_manager.cc:1398] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:03.867985 3963 tablet_bootstrap.cc:492] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:03.868935 3963 tablet_bootstrap.cc:654] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.870112 3963 tablet_bootstrap.cc:492] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:03.870198 3963 ts_tablet_manager.cc:1367] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:03.870584 3963 raft_consensus.cc:348] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:03.870673 3963 raft_consensus.cc:374] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.870699 3963 raft_consensus.cc:729] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.870805 3963 consensus_queue.cc:260] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:03.871070 3963 ts_tablet_manager.cc:1398] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.872689 3961 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:03.872892 3961 ts_tablet_manager.cc:1367] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:03.873338 3961 raft_consensus.cc:348] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.873489 3961 raft_consensus.cc:374] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.873536 3961 raft_consensus.cc:729] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.873683 3961 consensus_queue.cc:260] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.873948 3961 ts_tablet_manager.cc:1398] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.874130 3961 tablet_bootstrap.cc:492] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:03.874979 3961 tablet_bootstrap.cc:654] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.876034 3961 tablet_bootstrap.cc:492] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:03.876108 3961 ts_tablet_manager.cc:1367] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:03.876534 3961 raft_consensus.cc:348] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:03.876642 3961 raft_consensus.cc:374] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.876672 3961 raft_consensus.cc:729] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.876785 3961 consensus_queue.cc:260] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:03.877058 3961 ts_tablet_manager.cc:1398] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:03.880448 3962 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:03.886842 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:03.890170 3752 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:03.890264 3752 ts_tablet_manager.cc:1367] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.055s user 0.001s sys 0.000s
I20240502 01:59:03.890702 3752 raft_consensus.cc:348] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.890808 3752 raft_consensus.cc:374] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.890833 3752 raft_consensus.cc:729] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.890941 3752 consensus_queue.cc:260] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.891187 3752 ts_tablet_manager.cc:1398] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:03.897256 3962 tablet_bootstrap.cc:654] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:03.900915 3966 tablet_bootstrap.cc:492] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:03.901000 3966 ts_tablet_manager.cc:1367] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.000s
I20240502 01:59:03.901453 3966 raft_consensus.cc:348] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:03.901553 3966 raft_consensus.cc:374] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.901579 3966 raft_consensus.cc:729] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.901685 3966 consensus_queue.cc:260] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:03.901930 3966 ts_tablet_manager.cc:1398] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.907755 3885 raft_consensus.cc:482] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:03.907860 3885 raft_consensus.cc:504] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.908188 3885 leader_election.cc:288] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:03.908849 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213c37d37f1c4a3c8fbc48deafa9edac" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:03.908977 29216 raft_consensus.cc:2454] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:03.909236 26594 leader_election.cc:302] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:03.909487 3885 raft_consensus.cc:2781] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:03.909543 3885 raft_consensus.cc:482] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:03.909579 3885 raft_consensus.cc:3037] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.910614 3885 raft_consensus.cc:504] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.910935 3885 leader_election.cc:288] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:03.911098 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213c37d37f1c4a3c8fbc48deafa9edac" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:03.911233 27064 raft_consensus.cc:2454] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:03.911413 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213c37d37f1c4a3c8fbc48deafa9edac" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:03.911525 27063 raft_consensus.cc:3037] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.912003 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213c37d37f1c4a3c8fbc48deafa9edac" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:03.912099 29216 raft_consensus.cc:3037] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:03.912515 27063 raft_consensus.cc:2454] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:03.912905 26595 leader_election.cc:302] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:03.912958 29216 raft_consensus.cc:2454] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:03.913247 3676 raft_consensus.cc:2781] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:03.913326 3676 raft_consensus.cc:686] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:03.913434 3676 consensus_queue.cc:237] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:03.916049 3962 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:03.916211 3962 ts_tablet_manager.cc:1367] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240502 01:59:03.916718 3962 raft_consensus.cc:348] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.916891 3962 raft_consensus.cc:374] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:03.916975 3962 raft_consensus.cc:729] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:03.917189 3962 consensus_queue.cc:260] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:03.917191 26514 catalog_manager.cc:5484] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:03.917742 3962 ts_tablet_manager.cc:1398] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:03.937978 26793 consensus_queue.cc:237] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:03.939409 26659 raft_consensus.cc:1261] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:03.939406 29489 raft_consensus.cc:1261] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:03.939747 3489 consensus_queue.cc:1035] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:03.939937 3252 consensus_queue.cc:1035] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:03.941751 3489 raft_consensus.cc:2932] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.942088 26659 raft_consensus.cc:2932] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.944314 29489 raft_consensus.cc:2932] T 052566382186437e8e0245de9f60c30c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.944823 26514 catalog_manager.cc:5484] T 052566382186437e8e0245de9f60c30c P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 4 to 5, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:03.948086 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 052566382186437e8e0245de9f60c30c with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:03.948452 3999 tablet_replica.cc:330] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:03.949998 3999 raft_consensus.cc:2229] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:03.950198 3999 raft_consensus.cc:2258] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:03.950959 3999 ts_tablet_manager.cc:1875] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:03.952118 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:03.953274 3999 ts_tablet_manager.cc:1888] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:59:03.953351 3999 log.cc:1196] T 052566382186437e8e0245de9f60c30c P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/052566382186437e8e0245de9f60c30c
I20240502 01:59:03.953917 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 052566382186437e8e0245de9f60c30c (table rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]) successfully deleted
I20240502 01:59:03.956772 3909 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:03.957204 3912 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:03.959291 3911 tablet.cc:1722] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:03.962142 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=b567d66ec9b241a8b2f8568d554d9492] alter complete (version 3)
I20240502 01:59:03.995250 29080 consensus_queue.cc:237] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:03.996733 29489 raft_consensus.cc:1261] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:03.996798 29216 raft_consensus.cc:1261] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:03.997238 3794 consensus_queue.cc:1035] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:03.997673 29016 consensus_peers.cc:485] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2563da5db7a4e5197532b31e9aef636. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:03.997907 3422 consensus_queue.cc:1035] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:03.999581 3422 raft_consensus.cc:2932] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:03.999969 29216 raft_consensus.cc:2932] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:04.001999 29489 raft_consensus.cc:2932] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:04.002481 26510 catalog_manager.cc:5484] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:04.014847 29216 raft_consensus.cc:1203] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:59:04.027449 3306 raft_consensus.cc:482] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.027539 3306 raft_consensus.cc:504] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.027944 3306 leader_election.cc:288] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:04.028398 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dd23a3674d4beea982741db70f16f8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:04.028515 29080 raft_consensus.cc:2454] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:04.028821 26592 leader_election.cc:302] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:04.029289 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dd23a3674d4beea982741db70f16f8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:04.029414 29349 raft_consensus.cc:2454] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:04.029852 3306 raft_consensus.cc:2781] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.029914 3306 raft_consensus.cc:482] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.029954 3306 raft_consensus.cc:3037] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.030953 3306 raft_consensus.cc:504] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.031273 3306 leader_election.cc:288] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:04.031675 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dd23a3674d4beea982741db70f16f8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:04.031700 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dd23a3674d4beea982741db70f16f8" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:04.031780 29080 raft_consensus.cc:3037] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.031812 29349 raft_consensus.cc:3037] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.032507 29080 raft_consensus.cc:2454] T 39dd23a3674d4beea982741db70f16f8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:04.032588 29349 raft_consensus.cc:2454] T 39dd23a3674d4beea982741db70f16f8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:04.032881 26592 leader_election.cc:302] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:04.033062 3306 raft_consensus.cc:2781] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.033193 3306 raft_consensus.cc:686] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:04.033313 3306 consensus_queue.cc:237] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.035374 26514 catalog_manager.cc:5484] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:04.062331 3372 raft_consensus.cc:482] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.062429 3372 raft_consensus.cc:504] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.062839 3372 leader_election.cc:288] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:04.063102 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab7a9b176213400f990c4d5df0d7567f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:04.063114 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab7a9b176213400f990c4d5df0d7567f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:04.063226 26930 raft_consensus.cc:2454] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:04.063226 29349 raft_consensus.cc:2454] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:04.063483 26995 leader_election.cc:302] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:04.063634 3372 raft_consensus.cc:2781] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.063684 3372 raft_consensus.cc:482] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.063726 3372 raft_consensus.cc:3037] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.064707 3372 raft_consensus.cc:504] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.065058 3372 leader_election.cc:288] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:04.065431 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab7a9b176213400f990c4d5df0d7567f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:04.065450 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab7a9b176213400f990c4d5df0d7567f" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:04.065541 29349 raft_consensus.cc:3037] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.065541 26930 raft_consensus.cc:3037] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.066445 26930 raft_consensus.cc:2454] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:04.066457 29349 raft_consensus.cc:2454] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:04.066777 26995 leader_election.cc:302] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:04.067231 3372 raft_consensus.cc:2781] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.067319 3372 raft_consensus.cc:686] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:04.067437 3372 consensus_queue.cc:237] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.068215 26513 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:04.068323 26513 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:04.069744 26506 catalog_manager.cc:5484] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:04.070236 26513 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 02aeb9bbfe404f8587bcdc97b47eb006
I20240502 01:59:04.070780 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 02aeb9bbfe404f8587bcdc97b47eb006 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:04 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:04.070950 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 02aeb9bbfe404f8587bcdc97b47eb006 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:04 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:04.071100 3999 tablet_replica.cc:330] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:04.071230 3999 raft_consensus.cc:2229] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:04.071327 3999 raft_consensus.cc:2258] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.071821 4072 tablet_replica.cc:330] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:04.071940 3999 ts_tablet_manager.cc:1875] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:04.072000 4072 raft_consensus.cc:2229] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:04.072187 4072 raft_consensus.cc:2258] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.073976 3999 ts_tablet_manager.cc:1888] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:04.074059 4072 ts_tablet_manager.cc:1875] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:04.074800 3999 log.cc:1196] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/02aeb9bbfe404f8587bcdc97b47eb006
I20240502 01:59:04.075085 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 02aeb9bbfe404f8587bcdc97b47eb006 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:04 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:04.075352 3999 ts_tablet_manager.cc:1909] T 02aeb9bbfe404f8587bcdc97b47eb006 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:04.075945 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 02aeb9bbfe404f8587bcdc97b47eb006 (table rebalancer_dynamic_table_new_name [id=b567d66ec9b241a8b2f8568d554d9492]) successfully deleted
I20240502 01:59:04.076503 4073 tablet_replica.cc:330] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:04.076689 4073 raft_consensus.cc:2229] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:04.076856 4073 raft_consensus.cc:2258] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.076946 4072 ts_tablet_manager.cc:1888] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:04.077044 4072 log.cc:1196] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/02aeb9bbfe404f8587bcdc97b47eb006
I20240502 01:59:04.077461 4073 ts_tablet_manager.cc:1875] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:04.077908 4072 ts_tablet_manager.cc:1909] T 02aeb9bbfe404f8587bcdc97b47eb006 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:04.078608 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 02aeb9bbfe404f8587bcdc97b47eb006 (table rebalancer_dynamic_table_new_name [id=b567d66ec9b241a8b2f8568d554d9492]) successfully deleted
I20240502 01:59:04.080024 4073 ts_tablet_manager.cc:1888] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:04.080180 4073 log.cc:1196] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/02aeb9bbfe404f8587bcdc97b47eb006
I20240502 01:59:04.081177 4073 ts_tablet_manager.cc:1909] T 02aeb9bbfe404f8587bcdc97b47eb006 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:04.082485 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 02aeb9bbfe404f8587bcdc97b47eb006 (table rebalancer_dynamic_table_new_name [id=b567d66ec9b241a8b2f8568d554d9492]) successfully deleted
I20240502 01:59:04.111439 29330 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:04.112463 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:04.120541 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:04.121711 27044 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:04.132134 29059 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:04.137976 26640 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:04.139513 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:04.144658 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:04.173321 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:04.178719 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd1427b1a58494c949abe090a77e734 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265]), partition=
I20240502 01:59:04.178967 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd1427b1a58494c949abe090a77e734. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.178993 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd1427b1a58494c949abe090a77e734 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265]), partition=
I20240502 01:59:04.179158 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd1427b1a58494c949abe090a77e734 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265]), partition=
I20240502 01:59:04.179217 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd1427b1a58494c949abe090a77e734. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.179359 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd1427b1a58494c949abe090a77e734. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.181380 3962 tablet_bootstrap.cc:492] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:04.181419 3752 tablet_bootstrap.cc:492] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:04.182202 3752 tablet_bootstrap.cc:654] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.182294 3962 tablet_bootstrap.cc:654] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.183454 3962 tablet_bootstrap.cc:492] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:04.183537 3962 ts_tablet_manager.cc:1367] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:04.183924 3962 raft_consensus.cc:348] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.184027 3962 raft_consensus.cc:374] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.184054 3962 raft_consensus.cc:729] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.184172 3962 consensus_queue.cc:260] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.184417 3962 ts_tablet_manager.cc:1398] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.186407 4085 tablet_bootstrap.cc:492] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:04.187316 4085 tablet_bootstrap.cc:654] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.188331 3752 tablet_bootstrap.cc:492] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:04.188443 3752 ts_tablet_manager.cc:1367] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:04.188868 3752 raft_consensus.cc:348] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.188966 3752 raft_consensus.cc:374] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.188999 3752 raft_consensus.cc:729] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.189111 3752 consensus_queue.cc:260] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.189368 4085 tablet_bootstrap.cc:492] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:04.189400 3752 ts_tablet_manager.cc:1398] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:04.189443 4085 ts_tablet_manager.cc:1367] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:04.189846 4085 raft_consensus.cc:348] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.189951 4085 raft_consensus.cc:374] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.189980 4085 raft_consensus.cc:729] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.190127 4085 consensus_queue.cc:260] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.190666 4085 ts_tablet_manager.cc:1398] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.209411 3252 raft_consensus.cc:482] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.209513 3252 raft_consensus.cc:504] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.209874 3252 leader_election.cc:288] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:04.210222 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd1427b1a58494c949abe090a77e734" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:04.210335 27063 raft_consensus.cc:2454] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:04.210666 26730 leader_election.cc:302] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:04.210922 3252 raft_consensus.cc:2781] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.210922 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd1427b1a58494c949abe090a77e734" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:04.210979 3252 raft_consensus.cc:482] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.211015 3252 raft_consensus.cc:3037] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.211052 26930 raft_consensus.cc:2454] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:04.212002 3252 raft_consensus.cc:504] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.212347 3252 leader_election.cc:288] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:04.212607 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd1427b1a58494c949abe090a77e734" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:04.212671 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd1427b1a58494c949abe090a77e734" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:04.212723 27063 raft_consensus.cc:3037] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.212772 26930 raft_consensus.cc:3037] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.213575 27063 raft_consensus.cc:2454] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:04.213595 26930 raft_consensus.cc:2454] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:04.213874 26730 leader_election.cc:302] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:04.214143 3252 raft_consensus.cc:2781] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.214228 3252 raft_consensus.cc:686] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:04.214344 3252 consensus_queue.cc:237] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:04.216192 26511 catalog_manager.cc:5484] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:04.310689 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:04.321731 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:04.324713 29330 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:04.326578 27044 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:04.327343 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:04.334789 26640 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:04.336179 29059 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:04.342770 26774 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:04.344380 27063 raft_consensus.cc:1261] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:04.344746 3993 consensus_queue.cc:1035] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.345010 26930 raft_consensus.cc:1261] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:04.345371 3993 consensus_queue.cc:1035] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.347608 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:04.349179 26514 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet 9fd1427b1a58494c949abe090a77e734 (table rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265]). Expected version 1 got 0
I20240502 01:59:04.349828 26511 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 9fd1427b1a58494c949abe090a77e734 (table rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265]). Expected version 1 got 0
I20240502 01:59:04.357879 3912 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:04.358182 3912 tablet.cc:1620] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:04.365381 26513 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet 9fd1427b1a58494c949abe090a77e734 (table rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265]). Expected version 1 got 0
I20240502 01:59:04.365936 4146 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:04.366235 4146 tablet.cc:1620] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:04.371780 26514 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265] alter complete (version 1)
I20240502 01:59:04.372715 4147 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:04.373330 4147 tablet.cc:1620] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:04.418233 4157 ts_tablet_manager.cc:902] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:04.418848 4157 tablet_copy_client.cc:308] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:59:04.419397 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet c2563da5db7a4e5197532b31e9aef636 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:41275)
I20240502 01:59:04.419504 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet c2563da5db7a4e5197532b31e9aef636 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:41275: session id = 842c1b011b5d4c77bfd9b9591111502e-c2563da5db7a4e5197532b31e9aef636
I20240502 01:59:04.420464 29102 tablet_copy_source_session.cc:205] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:04.421280 4157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2563da5db7a4e5197532b31e9aef636. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.423579 4157 tablet_copy_client.cc:783] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:04.424026 4157 tablet_copy_client.cc:647] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:04.424324 3306 consensus_queue.cc:1035] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.426919 4157 tablet_copy_client.cc:523] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:04.427632 3676 consensus_queue.cc:1035] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.428237 4157 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:04.432320 4157 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:04.432570 4157 tablet_bootstrap.cc:492] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:04.432729 4157 ts_tablet_manager.cc:1367] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 01:59:04.433140 4157 raft_consensus.cc:348] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:04.433249 4157 raft_consensus.cc:729] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.433387 4157 consensus_queue.cc:260] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:04.433681 4157 ts_tablet_manager.cc:1398] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:04.434171 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-c2563da5db7a4e5197532b31e9aef636 received from {username='slave'} at 127.19.43.195:41275
I20240502 01:59:04.434240 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-c2563da5db7a4e5197532b31e9aef636 on tablet c2563da5db7a4e5197532b31e9aef636 with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:04.438536 26793 consensus_queue.cc:237] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.439922 26659 raft_consensus.cc:1261] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:04.439913 29213 raft_consensus.cc:1261] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 01:59:04.440230 3252 consensus_queue.cc:1035] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:04.440407 3993 consensus_queue.cc:1035] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:04.442008 3489 raft_consensus.cc:2932] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:04.442412 26659 raft_consensus.cc:2932] T dcf3a090ea284d12979b161f59081ff7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:04.442518 29213 raft_consensus.cc:2932] T dcf3a090ea284d12979b161f59081ff7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:04.445017 26511 catalog_manager.cc:5484] T dcf3a090ea284d12979b161f59081ff7 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 4 to 5, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:04.448168 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet dcf3a090ea284d12979b161f59081ff7 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:04.448760 4175 tablet_replica.cc:330] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:04.448959 4175 raft_consensus.cc:2229] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:04.449160 4175 raft_consensus.cc:2258] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.449723 4175 ts_tablet_manager.cc:1875] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:04.452956 4175 ts_tablet_manager.cc:1888] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 01:59:04.453034 4175 log.cc:1196] T dcf3a090ea284d12979b161f59081ff7 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/dcf3a090ea284d12979b161f59081ff7
I20240502 01:59:04.453696 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet dcf3a090ea284d12979b161f59081ff7 (table rebalance_test_table_5 [id=68ae40c8c303404c8908467f3134217f]) successfully deleted
I20240502 01:59:04.474128 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:04.475657 3676 consensus_queue.cc:1035] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.479307 3306 consensus_queue.cc:1035] T 39dd23a3674d4beea982741db70f16f8 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.479588 4146 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:04.479879 4146 tablet.cc:1620] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:04.480726 4147 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:04.481001 4147 tablet.cc:1620] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:04.481685 3912 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:04.481947 3912 tablet.cc:1620] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:04.486259 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=55ebccdf98444f6680fe40e641a00265] alter complete (version 2)
I20240502 01:59:04.531862 3372 consensus_queue.cc:1035] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.534773 3372 consensus_queue.cc:1035] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:04.577113 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:04.581154 26908 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:04.581364 27044 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:04.583148 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:04.587033 26640 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:04.594218 29330 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:04.595746 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:04.597116 29059 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:04.604488 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:04.607756 4146 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:04.610646 3912 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:04.613725 4147 tablet.cc:1722] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:04.613915 26774 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:04.621379 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=55ebccdf98444f6680fe40e641a00265] alter complete (version 3)
I20240502 01:59:04.622985 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 273b6d65e3df42ffb072e2129e832a92 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.623050 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 273b6d65e3df42ffb072e2129e832a92 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.623250 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273b6d65e3df42ffb072e2129e832a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.623294 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273b6d65e3df42ffb072e2129e832a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.623859 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 273b6d65e3df42ffb072e2129e832a92 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.624090 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273b6d65e3df42ffb072e2129e832a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.625507 29329 tablet_service.cc:1467] Processing CreateTablet for tablet aa5b1e209f3c486e94bda3dd2999abd3 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.625710 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5b1e209f3c486e94bda3dd2999abd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.626585 29059 tablet_service.cc:1467] Processing CreateTablet for tablet aa5b1e209f3c486e94bda3dd2999abd3 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.626853 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5b1e209f3c486e94bda3dd2999abd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.628293 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 83db30db2e6f4346822d5d55a1ce1e8f (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.628507 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83db30db2e6f4346822d5d55a1ce1e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.630420 3962 tablet_bootstrap.cc:492] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:04.630780 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 83db30db2e6f4346822d5d55a1ce1e8f (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.631002 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83db30db2e6f4346822d5d55a1ce1e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.631335 3962 tablet_bootstrap.cc:654] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.633002 3752 tablet_bootstrap.cc:492] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:04.633422 4247 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:04.633831 3752 tablet_bootstrap.cc:654] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.634232 4247 tablet_bootstrap.cc:654] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.634662 3962 tablet_bootstrap.cc:492] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:04.634733 3962 ts_tablet_manager.cc:1367] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:04.634761 26640 tablet_service.cc:1467] Processing CreateTablet for tablet aa5b1e209f3c486e94bda3dd2999abd3 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.634999 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5b1e209f3c486e94bda3dd2999abd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.635102 3962 raft_consensus.cc:348] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.635188 3962 raft_consensus.cc:374] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.635213 3962 raft_consensus.cc:729] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.635319 3962 consensus_queue.cc:260] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.635453 4246 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:04.635591 3962 ts_tablet_manager.cc:1398] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.637456 4252 tablet_bootstrap.cc:492] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:04.638285 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 83db30db2e6f4346822d5d55a1ce1e8f (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:04.638501 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83db30db2e6f4346822d5d55a1ce1e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.638631 4252 tablet_bootstrap.cc:654] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.639288 4245 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:04.640163 4245 tablet_bootstrap.cc:654] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.640818 3752 tablet_bootstrap.cc:492] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:04.640920 3752 ts_tablet_manager.cc:1367] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:59:04.641330 3752 raft_consensus.cc:348] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.641451 3752 raft_consensus.cc:374] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.641489 3752 raft_consensus.cc:729] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.641599 3752 consensus_queue.cc:260] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.641871 3752 ts_tablet_manager.cc:1398] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.641960 4245 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:04.642053 4245 ts_tablet_manager.cc:1367] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:04.642433 4245 raft_consensus.cc:348] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.642515 4245 raft_consensus.cc:374] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.642539 4245 raft_consensus.cc:729] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.642629 4245 consensus_queue.cc:260] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.642886 4245 ts_tablet_manager.cc:1398] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.643044 4245 tablet_bootstrap.cc:492] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:04.643344 4246 tablet_bootstrap.cc:654] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.643788 4248 tablet_bootstrap.cc:492] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:04.643980 4245 tablet_bootstrap.cc:654] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.645690 4248 tablet_bootstrap.cc:654] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.647457 4245 tablet_bootstrap.cc:492] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:04.647536 4245 ts_tablet_manager.cc:1367] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:04.647933 4245 raft_consensus.cc:348] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.648036 4245 raft_consensus.cc:374] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.648069 4245 raft_consensus.cc:729] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.648178 4245 consensus_queue.cc:260] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.648432 4245 ts_tablet_manager.cc:1398] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.659134 4248 tablet_bootstrap.cc:492] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:04.659335 4248 ts_tablet_manager.cc:1367] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 01:59:04.659835 4248 raft_consensus.cc:348] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.659997 4248 raft_consensus.cc:374] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.660077 4248 raft_consensus.cc:729] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.660238 4246 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:04.660373 4246 ts_tablet_manager.cc:1367] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 01:59:04.660295 4248 consensus_queue.cc:260] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.660861 4246 raft_consensus.cc:348] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.661005 4246 raft_consensus.cc:374] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.661072 4246 raft_consensus.cc:729] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.661240 4246 consensus_queue.cc:260] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.661736 4246 ts_tablet_manager.cc:1398] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.662478 4248 ts_tablet_manager.cc:1398] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:04.662681 4246 tablet_bootstrap.cc:492] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:04.663162 4247 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:04.663252 4247 ts_tablet_manager.cc:1367] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 01:59:04.663574 4246 tablet_bootstrap.cc:654] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.663650 4247 raft_consensus.cc:348] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.663748 4247 raft_consensus.cc:374] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.663784 4247 raft_consensus.cc:729] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.663897 4247 consensus_queue.cc:260] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.664147 4247 ts_tablet_manager.cc:1398] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.666913 4252 tablet_bootstrap.cc:492] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:04.666999 4252 ts_tablet_manager.cc:1367] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 01:59:04.667438 4252 raft_consensus.cc:348] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.667529 4252 raft_consensus.cc:374] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.667553 4252 raft_consensus.cc:729] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.667651 4252 consensus_queue.cc:260] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.667894 4252 ts_tablet_manager.cc:1398] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.673853 4246 tablet_bootstrap.cc:492] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:04.673938 4246 ts_tablet_manager.cc:1367] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:59:04.674350 4246 raft_consensus.cc:348] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.674453 4246 raft_consensus.cc:374] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.674516 4246 raft_consensus.cc:729] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.674683 4246 consensus_queue.cc:260] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.675065 4246 ts_tablet_manager.cc:1398] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:04.725123 3536 raft_consensus.cc:482] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.725229 3536 raft_consensus.cc:504] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.725693 3536 leader_election.cc:288] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:04.726049 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5b1e209f3c486e94bda3dd2999abd3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:04.726110 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5b1e209f3c486e94bda3dd2999abd3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:04.726190 26659 raft_consensus.cc:2454] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:04.726233 29080 raft_consensus.cc:2454] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:04.726610 29283 leader_election.cc:302] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:04.726779 3536 raft_consensus.cc:2781] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.726827 3536 raft_consensus.cc:482] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.726871 3536 raft_consensus.cc:3037] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.728034 3536 raft_consensus.cc:504] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.728344 3536 leader_election.cc:288] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:04.728741 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5b1e209f3c486e94bda3dd2999abd3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:04.728847 26659 raft_consensus.cc:3037] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.729256 26510 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:04.729350 26510 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:04.729844 26659 raft_consensus.cc:2454] T aa5b1e209f3c486e94bda3dd2999abd3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:04.730293 29285 leader_election.cc:302] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:04.731308 3536 raft_consensus.cc:2781] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.731353 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5b1e209f3c486e94bda3dd2999abd3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:04.731393 3536 raft_consensus.cc:686] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:04.731449 29080 raft_consensus.cc:3037] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.731515 3536 consensus_queue.cc:237] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:04.732328 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 9fd1427b1a58494c949abe090a77e734
I20240502 01:59:04.732424 29080 raft_consensus.cc:2454] T aa5b1e209f3c486e94bda3dd2999abd3 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:04.733161 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd1427b1a58494c949abe090a77e734 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:04 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:04.733340 4175 tablet_replica.cc:330] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:04.733347 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd1427b1a58494c949abe090a77e734 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:04 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:04.733484 4175 raft_consensus.cc:2229] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:04.733597 4175 raft_consensus.cc:2258] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.733608 4287 tablet_replica.cc:330] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:04.733744 4287 raft_consensus.cc:2229] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:04.734406 4175 ts_tablet_manager.cc:1875] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:04.734476 4287 raft_consensus.cc:2258] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.735328 26512 catalog_manager.cc:5484] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:04.736357 4287 ts_tablet_manager.cc:1875] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:04.736533 4175 ts_tablet_manager.cc:1888] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:04.736598 4175 log.cc:1196] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/9fd1427b1a58494c949abe090a77e734
I20240502 01:59:04.736830 4175 ts_tablet_manager.cc:1909] T 9fd1427b1a58494c949abe090a77e734 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:59:04.737250 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd1427b1a58494c949abe090a77e734 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:04 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:04.737946 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 9fd1427b1a58494c949abe090a77e734 (table rebalancer_dynamic_table_new_name [id=55ebccdf98444f6680fe40e641a00265]) successfully deleted
I20240502 01:59:04.738307 4308 tablet_replica.cc:330] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:04.739111 4308 raft_consensus.cc:2229] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:04.739391 4308 raft_consensus.cc:2258] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:04.740041 4308 ts_tablet_manager.cc:1875] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:04.740842 4287 ts_tablet_manager.cc:1888] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:04.740912 4287 log.cc:1196] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/9fd1427b1a58494c949abe090a77e734
I20240502 01:59:04.741199 4287 ts_tablet_manager.cc:1909] T 9fd1427b1a58494c949abe090a77e734 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:04.741860 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 9fd1427b1a58494c949abe090a77e734 (table rebalancer_dynamic_table_new_name [id=55ebccdf98444f6680fe40e641a00265]) successfully deleted
I20240502 01:59:04.742923 4308 ts_tablet_manager.cc:1888] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:04.743095 4308 log.cc:1196] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/9fd1427b1a58494c949abe090a77e734
I20240502 01:59:04.743463 4308 ts_tablet_manager.cc:1909] T 9fd1427b1a58494c949abe090a77e734 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:04.747238 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 9fd1427b1a58494c949abe090a77e734 (table rebalancer_dynamic_table_new_name [id=55ebccdf98444f6680fe40e641a00265]) successfully deleted
I20240502 01:59:04.770200 3899 raft_consensus.cc:482] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.770308 3899 raft_consensus.cc:504] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.770699 3899 leader_election.cc:288] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:04.771250 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83db30db2e6f4346822d5d55a1ce1e8f" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:04.771382 29489 raft_consensus.cc:2454] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:04.771613 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83db30db2e6f4346822d5d55a1ce1e8f" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:04.771661 26862 leader_election.cc:302] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:04.771730 27064 raft_consensus.cc:2454] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:04.772203 3899 raft_consensus.cc:2781] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.772258 3899 raft_consensus.cc:482] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.772295 3899 raft_consensus.cc:3037] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.773072 3899 raft_consensus.cc:504] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.773386 3899 leader_election.cc:288] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:04.774456 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83db30db2e6f4346822d5d55a1ce1e8f" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:04.774560 27064 raft_consensus.cc:3037] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.775498 27064 raft_consensus.cc:2454] T 83db30db2e6f4346822d5d55a1ce1e8f P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:04.775857 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83db30db2e6f4346822d5d55a1ce1e8f" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:04.775979 29489 raft_consensus.cc:3037] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.776921 29489 raft_consensus.cc:2454] T 83db30db2e6f4346822d5d55a1ce1e8f P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:04.777542 26862 leader_election.cc:302] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:04.777937 3124 raft_consensus.cc:2781] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.778009 3124 raft_consensus.cc:686] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:04.778158 3124 consensus_queue.cc:237] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:04.782680 26511 catalog_manager.cc:5484] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:04.836741 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:04.842453 26774 tablet_service.cc:1467] Processing CreateTablet for tablet d2244ec54f4643f0bfea714cda640399 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0]), partition=
I20240502 01:59:04.842454 29059 tablet_service.cc:1467] Processing CreateTablet for tablet d2244ec54f4643f0bfea714cda640399 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0]), partition=
I20240502 01:59:04.842696 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2244ec54f4643f0bfea714cda640399. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.842698 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2244ec54f4643f0bfea714cda640399. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.844655 4248 tablet_bootstrap.cc:492] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:04.844959 4316 tablet_bootstrap.cc:492] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:04.845535 4248 tablet_bootstrap.cc:654] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.845532 29195 tablet_service.cc:1467] Processing CreateTablet for tablet d2244ec54f4643f0bfea714cda640399 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0]), partition=
I20240502 01:59:04.845717 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2244ec54f4643f0bfea714cda640399. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:04.845775 4316 tablet_bootstrap.cc:654] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.846985 4248 tablet_bootstrap.cc:492] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:04.847069 4248 ts_tablet_manager.cc:1367] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:04.847524 4247 tablet_bootstrap.cc:492] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:04.847472 4248 raft_consensus.cc:348] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.847569 4248 raft_consensus.cc:374] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.847596 4248 raft_consensus.cc:729] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.847705 4248 consensus_queue.cc:260] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.847992 4248 ts_tablet_manager.cc:1398] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.848346 4247 tablet_bootstrap.cc:654] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:04.849151 4316 tablet_bootstrap.cc:492] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:04.849237 4316 ts_tablet_manager.cc:1367] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:04.849495 4247 tablet_bootstrap.cc:492] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:04.849587 4247 ts_tablet_manager.cc:1367] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:04.849591 4316 raft_consensus.cc:348] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.849704 4316 raft_consensus.cc:374] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.849767 4316 raft_consensus.cc:729] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.849910 4316 consensus_queue.cc:260] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.850087 4247 raft_consensus.cc:348] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.850212 4247 raft_consensus.cc:374] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:04.850266 4247 raft_consensus.cc:729] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:04.850329 4316 ts_tablet_manager.cc:1398] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:04.850387 4247 consensus_queue.cc:260] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.850662 4247 ts_tablet_manager.cc:1398] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:04.918069 3794 raft_consensus.cc:482] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.918187 3794 raft_consensus.cc:504] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.918668 3794 leader_election.cc:288] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:04.919051 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2244ec54f4643f0bfea714cda640399" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:04.919178 26793 raft_consensus.cc:2454] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:04.919173 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2244ec54f4643f0bfea714cda640399" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:04.919296 29216 raft_consensus.cc:2454] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:04.919489 29015 leader_election.cc:302] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:04.919644 3794 raft_consensus.cc:2781] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.919700 3794 raft_consensus.cc:482] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.919731 3794 raft_consensus.cc:3037] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.920778 3794 raft_consensus.cc:504] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.921094 3794 leader_election.cc:288] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:04.921422 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2244ec54f4643f0bfea714cda640399" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:04.921526 26793 raft_consensus.cc:3037] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.922144 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2244ec54f4643f0bfea714cda640399" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:04.922246 29216 raft_consensus.cc:3037] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.922443 26793 raft_consensus.cc:2454] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:04.922719 29015 leader_election.cc:302] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:04.922883 3794 raft_consensus.cc:2781] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.922966 3794 raft_consensus.cc:686] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:04.923076 3794 consensus_queue.cc:237] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:04.923139 29216 raft_consensus.cc:2454] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:04.925267 26511 catalog_manager.cc:5484] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:04.943466 3536 raft_consensus.cc:482] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:04.943557 3536 raft_consensus.cc:504] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.943961 3536 leader_election.cc:288] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:04.944226 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:04.944233 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:04.944353 29489 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:04.944345 29216 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:04.944697 29286 leader_election.cc:302] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:04.944845 3536 raft_consensus.cc:2781] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:04.944898 3536 raft_consensus.cc:482] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:04.944932 3536 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.945807 3536 raft_consensus.cc:504] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.946149 3536 leader_election.cc:288] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:04.946575 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:04.946671 29489 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.947428 29489 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:04.947657 29286 leader_election.cc:302] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:04.947801 3536 raft_consensus.cc:2781] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:04.947876 3536 raft_consensus.cc:686] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:04.947999 3536 consensus_queue.cc:237] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:04.948455 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:04.948580 29213 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:04.949476 29213 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:04.950071 26511 catalog_manager.cc:5484] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:04.974181 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:04.976823 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:04.982280 27044 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:04.983726 26930 raft_consensus.cc:1203] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:04.986857 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:04.990406 29329 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:04.993644 26640 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:05.002171 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:05.015719 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:05.046231 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:05.050164 29216 raft_consensus.cc:1261] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:05.050197 26793 raft_consensus.cc:1261] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:05.050571 3794 consensus_queue.cc:1035] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.050731 2712 consensus_queue.cc:1035] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.053253 26511 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet d2244ec54f4643f0bfea714cda640399 (table rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0]). Expected version 1 got 0
I20240502 01:59:05.054086 4146 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:05.054390 4146 tablet.cc:1620] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:05.055016 26515 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet d2244ec54f4643f0bfea714cda640399 (table rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0]). Expected version 1 got 0
I20240502 01:59:05.055598 4340 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:05.056001 4340 tablet.cc:1620] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:05.057744 4339 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:05.058351 4339 tablet.cc:1620] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:05.058924 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0] alter complete (version 1)
I20240502 01:59:05.138641 3536 consensus_queue.cc:1035] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.142849 3421 consensus_queue.cc:1035] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.163924 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:05.168787 4339 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:05.169060 4339 tablet.cc:1620] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:05.169824 4340 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:05.169939 4146 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:05.170222 4340 tablet.cc:1620] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:05.170275 4146 tablet.cc:1620] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:05.174834 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=0488bc8d7488467ca4aaa3389f87a4a0] alter complete (version 2)
I20240502 01:59:05.252202 3899 consensus_queue.cc:1035] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.255926 3899 consensus_queue.cc:1035] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.278926 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:05.283725 4339 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:05.284312 4340 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:05.285194 4146 tablet.cc:1722] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:05.294953 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=0488bc8d7488467ca4aaa3389f87a4a0] alter complete (version 3)
I20240502 01:59:05.338199 3536 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.342392 3421 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.368827 29329 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:05.368959 27044 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:05.377266 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:05.379840 29468 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:05.390177 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:05.391573 26774 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:05.395812 26640 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:05.406318 26508 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:05.406459 26508 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:05.408701 26508 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet d2244ec54f4643f0bfea714cda640399
I20240502 01:59:05.409202 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:05.409507 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet d2244ec54f4643f0bfea714cda640399 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:05 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:05.409513 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet d2244ec54f4643f0bfea714cda640399 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:05 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:05.409862 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet d2244ec54f4643f0bfea714cda640399 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:05 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:05.412040 4409 tablet_replica.cc:330] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:05.412217 4409 raft_consensus.cc:2229] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:05.412492 4409 raft_consensus.cc:2258] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:05.413116 4409 ts_tablet_manager.cc:1875] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:05.427801 4407 tablet_replica.cc:330] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:05.427975 4407 raft_consensus.cc:2229] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:05.428577 4407 raft_consensus.cc:2258] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:05.429708 4407 ts_tablet_manager.cc:1875] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:05.432193 4408 tablet_replica.cc:330] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:05.432374 4408 raft_consensus.cc:2229] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:05.432483 4408 raft_consensus.cc:2258] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:05.433038 4408 ts_tablet_manager.cc:1875] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:05.433135 4409 ts_tablet_manager.cc:1888] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:05.433239 4409 log.cc:1196] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/d2244ec54f4643f0bfea714cda640399
I20240502 01:59:05.433815 4409 ts_tablet_manager.cc:1909] T d2244ec54f4643f0bfea714cda640399 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:05.434571 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet d2244ec54f4643f0bfea714cda640399 (table rebalancer_dynamic_table_new_name [id=0488bc8d7488467ca4aaa3389f87a4a0]) successfully deleted
I20240502 01:59:05.436743 4408 ts_tablet_manager.cc:1888] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:05.436806 4408 log.cc:1196] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/d2244ec54f4643f0bfea714cda640399
I20240502 01:59:05.437039 4408 ts_tablet_manager.cc:1909] T d2244ec54f4643f0bfea714cda640399 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:05.437698 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet d2244ec54f4643f0bfea714cda640399 (table rebalancer_dynamic_table_new_name [id=0488bc8d7488467ca4aaa3389f87a4a0]) successfully deleted
I20240502 01:59:05.438223 4407 ts_tablet_manager.cc:1888] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:05.438295 4407 log.cc:1196] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/d2244ec54f4643f0bfea714cda640399
I20240502 01:59:05.438524 4407 ts_tablet_manager.cc:1909] T d2244ec54f4643f0bfea714cda640399 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:05.439574 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet d2244ec54f4643f0bfea714cda640399 (table rebalancer_dynamic_table_new_name [id=0488bc8d7488467ca4aaa3389f87a4a0]) successfully deleted
I20240502 01:59:05.463280 29079 consensus_queue.cc:237] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.464557 26930 raft_consensus.cc:1261] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:05.464864 29216 raft_consensus.cc:1261] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:05.465142 3794 consensus_queue.cc:1035] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:05.465328 2712 consensus_queue.cc:1035] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:05.467224 4338 raft_consensus.cc:2932] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:05.467367 29216 raft_consensus.cc:2932] T c2563da5db7a4e5197532b31e9aef636 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:05.467386 26930 raft_consensus.cc:2932] T c2563da5db7a4e5197532b31e9aef636 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:05.470314 26508 catalog_manager.cc:5484] T c2563da5db7a4e5197532b31e9aef636 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:05.472985 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet c2563da5db7a4e5197532b31e9aef636 with delete_type TABLET_DATA_TOMBSTONED (TS 7c693d0933c64d9babc07f404df45cd1 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:05.473353 4429 tablet_replica.cc:330] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:05.474581 4429 raft_consensus.cc:2229] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:05.474766 4429 raft_consensus.cc:2258] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:05.475122 4429 ts_tablet_manager.cc:1875] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:05.476816 4429 ts_tablet_manager.cc:1888] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:05.476876 4429 log.cc:1196] T c2563da5db7a4e5197532b31e9aef636 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/c2563da5db7a4e5197532b31e9aef636
I20240502 01:59:05.477274 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet c2563da5db7a4e5197532b31e9aef636 (table rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]) successfully deleted
I20240502 01:59:05.500881 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:05.506392 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8b72ba18fd4d21bd3c8bb2a1efd18c (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.506693 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8b72ba18fd4d21bd3c8bb2a1efd18c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.506964 27044 tablet_service.cc:1467] Processing CreateTablet for tablet cbeef9995d6e46dfbbff976d0bb9028e (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.507226 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbeef9995d6e46dfbbff976d0bb9028e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.507280 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8b72ba18fd4d21bd3c8bb2a1efd18c (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.507484 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8b72ba18fd4d21bd3c8bb2a1efd18c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.509606 4431 tablet_bootstrap.cc:492] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:05.509588 26775 tablet_service.cc:1467] Processing CreateTablet for tablet cbeef9995d6e46dfbbff976d0bb9028e (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.509687 29454 tablet_service.cc:1467] Processing CreateTablet for tablet cbeef9995d6e46dfbbff976d0bb9028e (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.509810 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbeef9995d6e46dfbbff976d0bb9028e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.509934 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbeef9995d6e46dfbbff976d0bb9028e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.510174 4430 tablet_bootstrap.cc:492] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:05.510576 4431 tablet_bootstrap.cc:654] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.511152 4430 tablet_bootstrap.cc:654] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.511833 29195 tablet_service.cc:1467] Processing CreateTablet for tablet ab01aa0450bb4ef388b49e38369ab20b (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.512050 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab01aa0450bb4ef388b49e38369ab20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.512329 29059 tablet_service.cc:1467] Processing CreateTablet for tablet ab01aa0450bb4ef388b49e38369ab20b (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.512567 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab01aa0450bb4ef388b49e38369ab20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.512542 4430 tablet_bootstrap.cc:492] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:05.512662 4430 ts_tablet_manager.cc:1367] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:05.513075 4430 raft_consensus.cc:348] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.513165 4430 raft_consensus.cc:374] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.513190 4430 raft_consensus.cc:729] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.513423 4430 consensus_queue.cc:260] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.514431 4435 tablet_bootstrap.cc:492] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:05.514452 4430 ts_tablet_manager.cc:1398] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:05.514529 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:05.514803 4437 tablet_bootstrap.cc:492] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:05.515872 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8b72ba18fd4d21bd3c8bb2a1efd18c (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.516083 4431 tablet_bootstrap.cc:492] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:05.516101 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8b72ba18fd4d21bd3c8bb2a1efd18c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.516165 4431 ts_tablet_manager.cc:1367] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 01:59:05.516624 4431 raft_consensus.cc:348] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.516722 4431 raft_consensus.cc:374] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.516752 4431 raft_consensus.cc:729] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.516875 4431 consensus_queue.cc:260] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.517165 4431 ts_tablet_manager.cc:1398] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:05.518050 4437 tablet_bootstrap.cc:654] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.519958 4435 tablet_bootstrap.cc:654] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.520251 29454 tablet_service.cc:1467] Processing CreateTablet for tablet ab01aa0450bb4ef388b49e38369ab20b (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=0c9a4b7f62d147cdb650dbf3ec335502]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:05.520470 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab01aa0450bb4ef388b49e38369ab20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.521559 4435 tablet_bootstrap.cc:492] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:05.521629 4435 ts_tablet_manager.cc:1367] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:05.521713 27044 tablet_service.cc:1467] Processing CreateTablet for tablet d3c46255597d449ebb64f8e8d598ddba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]), partition=
I20240502 01:59:05.521898 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c46255597d449ebb64f8e8d598ddba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.522063 4435 raft_consensus.cc:348] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.522152 4435 raft_consensus.cc:374] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.522181 4435 raft_consensus.cc:729] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.522557 29059 tablet_service.cc:1467] Processing CreateTablet for tablet d3c46255597d449ebb64f8e8d598ddba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]), partition=
I20240502 01:59:05.522281 4435 consensus_queue.cc:260] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.522747 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c46255597d449ebb64f8e8d598ddba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.523046 4435 ts_tablet_manager.cc:1398] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:05.523335 4435 tablet_bootstrap.cc:492] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:05.523767 4431 tablet_bootstrap.cc:492] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:05.524425 4435 tablet_bootstrap.cc:654] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.524585 4431 tablet_bootstrap.cc:654] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.525698 4431 tablet_bootstrap.cc:492] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:05.525792 4431 ts_tablet_manager.cc:1367] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:59:05.526230 4431 raft_consensus.cc:348] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.526314 4431 raft_consensus.cc:374] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.526309 4436 tablet_bootstrap.cc:492] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:05.526340 4431 raft_consensus.cc:729] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.526449 4431 consensus_queue.cc:260] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.527383 4436 tablet_bootstrap.cc:654] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.530261 4432 tablet_bootstrap.cc:492] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:05.533179 4432 tablet_bootstrap.cc:654] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.533962 4431 ts_tablet_manager.cc:1398] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:59:05.534462 26908 tablet_service.cc:1467] Processing CreateTablet for tablet d3c46255597d449ebb64f8e8d598ddba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]), partition=
I20240502 01:59:05.534657 4437 tablet_bootstrap.cc:492] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:05.534706 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c46255597d449ebb64f8e8d598ddba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:05.534739 4437 ts_tablet_manager.cc:1367] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 01:59:05.535136 4437 raft_consensus.cc:348] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.535223 4437 raft_consensus.cc:374] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.535248 4437 raft_consensus.cc:729] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.535352 4437 consensus_queue.cc:260] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.535426 4436 tablet_bootstrap.cc:492] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:05.535506 4436 ts_tablet_manager.cc:1367] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:59:05.535588 4437 ts_tablet_manager.cc:1398] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:05.535755 4437 tablet_bootstrap.cc:492] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:05.536576 4437 tablet_bootstrap.cc:654] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.536971 4436 raft_consensus.cc:348] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.537075 4436 raft_consensus.cc:374] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.537107 4436 raft_consensus.cc:729] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.537231 4436 consensus_queue.cc:260] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.537385 4435 tablet_bootstrap.cc:492] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:05.537457 4435 ts_tablet_manager.cc:1367] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:05.537504 4436 ts_tablet_manager.cc:1398] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:05.537536 4444 tablet_bootstrap.cc:492] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:05.537858 4435 raft_consensus.cc:348] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.537952 4435 raft_consensus.cc:374] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.537979 4435 raft_consensus.cc:729] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.538108 4435 consensus_queue.cc:260] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.538390 4435 ts_tablet_manager.cc:1398] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:05.538522 4435 tablet_bootstrap.cc:492] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:05.539403 4435 tablet_bootstrap.cc:654] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.539415 4444 tablet_bootstrap.cc:654] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.544188 4435 tablet_bootstrap.cc:492] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:05.544267 4435 ts_tablet_manager.cc:1367] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:05.544584 4435 raft_consensus.cc:348] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.544667 4435 raft_consensus.cc:374] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.544700 4435 raft_consensus.cc:729] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.544787 4435 consensus_queue.cc:260] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.544986 4435 ts_tablet_manager.cc:1398] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:05.545584 4432 tablet_bootstrap.cc:492] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:05.545678 4432 ts_tablet_manager.cc:1367] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20240502 01:59:05.545859 4437 tablet_bootstrap.cc:492] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:05.545859 4444 tablet_bootstrap.cc:492] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:05.545930 4437 ts_tablet_manager.cc:1367] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 01:59:05.545931 4444 ts_tablet_manager.cc:1367] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:59:05.546268 4432 raft_consensus.cc:348] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.546375 4432 raft_consensus.cc:374] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.546403 4432 raft_consensus.cc:729] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.546356 4437 raft_consensus.cc:348] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.546353 4444 raft_consensus.cc:348] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.546443 4437 raft_consensus.cc:374] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.546443 4444 raft_consensus.cc:374] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.546483 4444 raft_consensus.cc:729] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.546543 4432 consensus_queue.cc:260] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.546599 4444 consensus_queue.cc:260] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.546864 4432 ts_tablet_manager.cc:1398] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:05.546872 4444 ts_tablet_manager.cc:1398] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:05.547029 4432 tablet_bootstrap.cc:492] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:05.547590 4437 raft_consensus.cc:729] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.547721 4437 consensus_queue.cc:260] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.547971 4437 ts_tablet_manager.cc:1398] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:05.548794 4432 tablet_bootstrap.cc:654] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:05.550267 4432 tablet_bootstrap.cc:492] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:05.551137 4432 ts_tablet_manager.cc:1367] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:05.551679 4432 raft_consensus.cc:348] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.551824 4432 raft_consensus.cc:374] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:05.551870 4432 raft_consensus.cc:729] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:05.551987 4432 consensus_queue.cc:260] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.552245 4432 ts_tablet_manager.cc:1398] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:05.583034 4410 raft_consensus.cc:482] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:05.583112 4410 raft_consensus.cc:504] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.583472 4410 leader_election.cc:288] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:05.583737 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:05.583868 29216 raft_consensus.cc:2454] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:05.583842 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:05.583953 29079 raft_consensus.cc:2454] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:05.584122 29421 leader_election.cc:302] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:05.584350 4410 raft_consensus.cc:2781] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:05.584414 4410 raft_consensus.cc:482] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:05.584457 4410 raft_consensus.cc:3037] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.585269 4410 raft_consensus.cc:504] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.585568 4410 leader_election.cc:288] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:05.585819 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:05.585909 29216 raft_consensus.cc:3037] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.586084 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:05.586185 29079 raft_consensus.cc:3037] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.586844 29216 raft_consensus.cc:2454] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:05.587005 29079 raft_consensus.cc:2454] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:05.587198 29421 leader_election.cc:302] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:05.587393 4410 raft_consensus.cc:2781] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:05.587453 4410 raft_consensus.cc:686] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:05.587559 4410 consensus_queue.cc:237] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.589671 26511 catalog_manager.cc:5484] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:05.615536 3794 raft_consensus.cc:482] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:05.615638 3794 raft_consensus.cc:504] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.615965 3794 leader_election.cc:288] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:05.616259 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c46255597d449ebb64f8e8d598ddba" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:05.616320 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c46255597d449ebb64f8e8d598ddba" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:05.616375 27064 raft_consensus.cc:2454] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:05.616451 26930 raft_consensus.cc:2454] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:05.616621 29016 leader_election.cc:302] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:05.616770 3794 raft_consensus.cc:2781] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:05.616835 3794 raft_consensus.cc:482] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:05.616881 3794 raft_consensus.cc:3037] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.617578 3794 raft_consensus.cc:504] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.617899 3794 leader_election.cc:288] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:05.618223 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c46255597d449ebb64f8e8d598ddba" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:05.618332 26930 raft_consensus.cc:3037] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.618464 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c46255597d449ebb64f8e8d598ddba" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:05.618570 27064 raft_consensus.cc:3037] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.619262 26930 raft_consensus.cc:2454] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:05.619529 27064 raft_consensus.cc:2454] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:05.619555 29016 leader_election.cc:302] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:05.619705 3794 raft_consensus.cc:2781] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:05.619807 3794 raft_consensus.cc:686] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:05.619930 3794 consensus_queue.cc:237] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:05.622056 26511 catalog_manager.cc:5484] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:59:05.624367 26842 tablet.cc:2369] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:59:05.629523 4345 raft_consensus.cc:482] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:05.629602 4345 raft_consensus.cc:504] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.629964 4345 leader_election.cc:288] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:05.630311 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeef9995d6e46dfbbff976d0bb9028e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:05.630362 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeef9995d6e46dfbbff976d0bb9028e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:05.630435 26793 raft_consensus.cc:2454] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:05.630475 29489 raft_consensus.cc:2454] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:05.630754 26998 leader_election.cc:302] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:05.631147 4345 raft_consensus.cc:2781] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:05.631206 4345 raft_consensus.cc:482] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:05.631249 4345 raft_consensus.cc:3037] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.632069 4345 raft_consensus.cc:504] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.632397 4345 leader_election.cc:288] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:05.632644 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeef9995d6e46dfbbff976d0bb9028e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:05.632750 29487 raft_consensus.cc:3037] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.632822 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeef9995d6e46dfbbff976d0bb9028e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:05.632938 26793 raft_consensus.cc:3037] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.633704 29487 raft_consensus.cc:2454] T cbeef9995d6e46dfbbff976d0bb9028e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:05.633836 26793 raft_consensus.cc:2454] T cbeef9995d6e46dfbbff976d0bb9028e P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:05.633970 26998 leader_election.cc:302] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:05.634159 4345 raft_consensus.cc:2781] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:05.634238 4345 raft_consensus.cc:686] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:05.634366 4345 consensus_queue.cc:237] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.636154 26511 catalog_manager.cc:5484] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:59:05.679776 27111 tablet.cc:2369] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:59:05.706663 26709 tablet.cc:2369] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:59:05.735451 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:05.739406 26930 raft_consensus.cc:1261] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:05.739481 27064 raft_consensus.cc:1261] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:05.739830 4338 consensus_queue.cc:1035] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.740134 4338 consensus_queue.cc:1035] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.744009 26513 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]). Expected version 1 got 0
I20240502 01:59:05.745936 26515 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]). Expected version 1 got 0
I20240502 01:59:05.747459 4339 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:05.747768 4339 tablet.cc:1620] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:05.747751 3676 raft_consensus.cc:482] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:05.747848 3676 raft_consensus.cc:504] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.748311 3676 leader_election.cc:288] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:05.748849 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8b72ba18fd4d21bd3c8bb2a1efd18c" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:05.748971 29487 raft_consensus.cc:2454] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:05.749294 26595 leader_election.cc:302] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:05.750105 26515 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]). Expected version 1 got 0
I20240502 01:59:05.753006 26513 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34]). Expected version 1 got 0
I20240502 01:59:05.753571 3676 raft_consensus.cc:2781] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:05.753630 3676 raft_consensus.cc:482] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:05.753665 3676 raft_consensus.cc:3037] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.753983 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34] alter complete (version 1)
I20240502 01:59:05.754765 3676 raft_consensus.cc:504] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.755085 3676 leader_election.cc:288] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:05.755944 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8b72ba18fd4d21bd3c8bb2a1efd18c" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:05.756057 29489 raft_consensus.cc:3037] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.756527 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8b72ba18fd4d21bd3c8bb2a1efd18c" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:05.756660 26794 raft_consensus.cc:3037] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:05.757055 29489 raft_consensus.cc:2454] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:05.757669 26794 raft_consensus.cc:2454] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:05.757865 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8b72ba18fd4d21bd3c8bb2a1efd18c" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:05.757987 26793 raft_consensus.cc:2362] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1. Re-sending same reply.
I20240502 01:59:05.758184 26595 leader_election.cc:302] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:05.760946 4497 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:05.761240 4497 tablet.cc:1620] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:05.762013 3676 raft_consensus.cc:2781] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:05.762120 3676 raft_consensus.cc:686] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:05.762252 3676 consensus_queue.cc:237] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:05.765013 26515 catalog_manager.cc:5484] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:05.766716 4496 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:05.767000 4496 tablet.cc:1620] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:05.832229 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:05.837584 26640 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:05.848292 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:05.849545 26775 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:05.849723 29329 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:05.850512 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:05.850510 29468 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:05.855196 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:05.862174 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:05.869829 4339 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:05.870146 4339 tablet.cc:1620] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:05.870467 4497 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:05.870711 4497 tablet.cc:1620] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:05.875453 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=1f6e598a2f7042f280bae679334fde34] alter complete (version 2)
I20240502 01:59:05.879840 4496 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:05.880100 4496 tablet.cc:1620] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:05.956485 29213 consensus_queue.cc:237] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:05.957942 29080 raft_consensus.cc:1261] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:05.957948 29350 raft_consensus.cc:1261] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:05.958459 3297 consensus_queue.cc:1035] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:05.958667 29149 consensus_peers.cc:485] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 499d2ca7bffc42748591db5280f829f2. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:05.958782 4527 consensus_queue.cc:1035] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:05.960711 4527 raft_consensus.cc:2932] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:05.961128 29080 raft_consensus.cc:2932] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:05.963053 29350 raft_consensus.cc:2932] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:05.965194 26507 catalog_manager.cc:5484] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:05.970633 4410 consensus_queue.cc:1035] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.974849 4005 consensus_queue.cc:1035] T ab01aa0450bb4ef388b49e38369ab20b P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:05.986413 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:05.993741 4339 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:05.994221 4496 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:05.994590 4497 tablet.cc:1722] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:06.000517 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=1f6e598a2f7042f280bae679334fde34] alter complete (version 3)
I20240502 01:59:06.006634 29350 consensus_queue.cc:237] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.007890 29487 raft_consensus.cc:1261] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.007892 29080 raft_consensus.cc:1261] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.008090 29286 consensus_peers.cc:485] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 089d9dc33a594712ac04ad28a0cd37b1. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.008282 3421 consensus_queue.cc:1035] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.008648 3421 consensus_queue.cc:1035] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.010244 4551 raft_consensus.cc:2932] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.010641 29080 raft_consensus.cc:2932] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.012259 29487 raft_consensus.cc:2932] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.013341 26515 catalog_manager.cc:5484] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.032737 29350 consensus_queue.cc:237] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.033965 29213 raft_consensus.cc:1261] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.034143 27064 raft_consensus.cc:1261] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.034413 4530 consensus_queue.cc:1035] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:06.034603 29285 consensus_peers.cc:485] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78609c45dc0c4e1b838bb737ebf7e034. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.034693 3421 consensus_queue.cc:1035] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.036257 3421 raft_consensus.cc:2932] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:06.036674 27064 raft_consensus.cc:2932] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:06.038955 26514 catalog_manager.cc:5484] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:06.039254 29213 raft_consensus.cc:2932] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:06.058893 29080 consensus_queue.cc:237] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.060063 29350 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 01:59:06.060124 27064 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 01:59:06.060438 29015 consensus_peers.cc:485] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8afd4c5362464c4cba43d5ef28b50c29. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.060474 4338 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:06.060629 3794 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:06.062372 4338 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.062758 29350 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.064138 27064 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.066606 26514 catalog_manager.cc:5484] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 3 to 4, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.083886 29350 consensus_queue.cc:237] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.085062 29213 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.085340 29285 consensus_peers.cc:485] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 273b6d65e3df42ffb072e2129e832a92. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.085458 29487 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.085523 4530 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.085806 4551 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.087668 4530 raft_consensus.cc:2932] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.088084 29487 raft_consensus.cc:2932] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.090323 29213 raft_consensus.cc:2932] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.091019 26507 catalog_manager.cc:5484] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.102581 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:06.102718 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:06.104570 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet d3c46255597d449ebb64f8e8d598ddba
I20240502 01:59:06.105244 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c46255597d449ebb64f8e8d598ddba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:06 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:06.105420 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c46255597d449ebb64f8e8d598ddba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:06 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:06.105518 4618 tablet_replica.cc:330] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:06.105644 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c46255597d449ebb64f8e8d598ddba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:06 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:06.105660 4618 raft_consensus.cc:2229] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:06.105777 4618 raft_consensus.cc:2258] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:06.105922 4620 tablet_replica.cc:330] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:06.105995 4619 tablet_replica.cc:330] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:06.106141 4620 raft_consensus.cc:2229] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:06.106165 4619 raft_consensus.cc:2229] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:06.106379 4619 raft_consensus.cc:2258] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:06.106379 4620 raft_consensus.cc:2258] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:06.106456 4618 ts_tablet_manager.cc:1875] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:06.106953 4619 ts_tablet_manager.cc:1875] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:06.107851 4620 ts_tablet_manager.cc:1875] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:06.108783 4618 ts_tablet_manager.cc:1888] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:06.108846 4618 log.cc:1196] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/d3c46255597d449ebb64f8e8d598ddba
I20240502 01:59:06.109120 4618 ts_tablet_manager.cc:1909] T d3c46255597d449ebb64f8e8d598ddba P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:06.109189 4619 ts_tablet_manager.cc:1888] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:06.109265 4619 log.cc:1196] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/d3c46255597d449ebb64f8e8d598ddba
I20240502 01:59:06.109580 4619 ts_tablet_manager.cc:1909] T d3c46255597d449ebb64f8e8d598ddba P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:59:06.109699 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table_new_name [id=1f6e598a2f7042f280bae679334fde34]) successfully deleted
I20240502 01:59:06.110263 4620 ts_tablet_manager.cc:1888] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:06.110272 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table_new_name [id=1f6e598a2f7042f280bae679334fde34]) successfully deleted
I20240502 01:59:06.110409 4620 log.cc:1196] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/d3c46255597d449ebb64f8e8d598ddba
I20240502 01:59:06.110656 4620 ts_tablet_manager.cc:1909] T d3c46255597d449ebb64f8e8d598ddba P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:06.111220 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet d3c46255597d449ebb64f8e8d598ddba (table rebalancer_dynamic_table_new_name [id=1f6e598a2f7042f280bae679334fde34]) successfully deleted
I20240502 01:59:06.113844 29487 consensus_queue.cc:237] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.115203 29349 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.115314 29422 consensus_peers.cc:485] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8209cefcc49042b18fdd97d845d167e2. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.115352 26659 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.115706 4410 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.115887 4627 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.117466 4627 raft_consensus.cc:2932] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.117864 29349 raft_consensus.cc:2932] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.119663 26659 raft_consensus.cc:2932] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.121126 26514 catalog_manager.cc:5484] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.143182 26659 consensus_queue.cc:237] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:06.144402 26930 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:06.144413 29487 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 01:59:06.144506 26592 consensus_peers.cc:485] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1cd4681de8645ff92b479ccdd3eac37. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.144765 3676 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.144945 3306 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.146584 3676 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:06.147006 29487 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:06.149839 26507 catalog_manager.cc:5484] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 5 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:06.150367 26930 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:06.169646 26930 consensus_queue.cc:237] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.171041 29080 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:06.171129 26659 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 01:59:06.171391 26861 consensus_peers.cc:485] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5515469fdf494900ae20f20fdfca587c. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.171718 3124 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.171936 4666 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.173439 3124 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.173965 26659 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.175485 29080 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.176285 26515 catalog_manager.cc:5484] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.194816 27064 consensus_queue.cc:237] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
W20240502 01:59:06.196358 26997 consensus_peers.cc:485] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab7a9b176213400f990c4d5df0d7567f. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.196884 26930 raft_consensus.cc:1261] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.197072 29349 raft_consensus.cc:1261] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.197365 3791 consensus_queue.cc:1035] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.197557 4451 consensus_queue.cc:1035] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.197783 4649 consensus_queue.cc:1035] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.199822 3791 raft_consensus.cc:2932] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.200255 26930 raft_consensus.cc:2932] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.201992 29349 raft_consensus.cc:2932] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.204391 4646 consensus_queue.cc:1035] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.205433 26515 catalog_manager.cc:5484] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.207105 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:06.214138 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 7317c932fdba4235816aff6d5c482878 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]), partition=
I20240502 01:59:06.214399 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7317c932fdba4235816aff6d5c482878. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.214429 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 7317c932fdba4235816aff6d5c482878 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]), partition=
I20240502 01:59:06.214650 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7317c932fdba4235816aff6d5c482878. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.216473 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 7317c932fdba4235816aff6d5c482878 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]), partition=
I20240502 01:59:06.216493 4697 tablet_bootstrap.cc:492] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:06.216684 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7317c932fdba4235816aff6d5c482878. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.217217 4698 tablet_bootstrap.cc:492] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:06.218564 4697 tablet_bootstrap.cc:654] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.219905 4698 tablet_bootstrap.cc:654] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.220064 4697 tablet_bootstrap.cc:492] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:06.220142 4697 ts_tablet_manager.cc:1367] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:06.221221 4698 tablet_bootstrap.cc:492] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:06.221226 4699 tablet_bootstrap.cc:492] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:06.221311 4698 ts_tablet_manager.cc:1367] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:06.221349 4697 raft_consensus.cc:348] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.221450 4697 raft_consensus.cc:374] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.221482 4697 raft_consensus.cc:729] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.221592 4697 consensus_queue.cc:260] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.221750 4698 raft_consensus.cc:348] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.221844 4698 raft_consensus.cc:374] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.221873 4698 raft_consensus.cc:729] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.221875 4697 ts_tablet_manager.cc:1398] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:06.221994 4698 consensus_queue.cc:260] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.222293 4698 ts_tablet_manager.cc:1398] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.222321 4699 tablet_bootstrap.cc:654] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.223542 4699 tablet_bootstrap.cc:492] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:06.223625 4699 ts_tablet_manager.cc:1367] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:06.224035 4699 raft_consensus.cc:348] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.224123 4699 raft_consensus.cc:374] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.224151 4699 raft_consensus.cc:729] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.224253 4699 consensus_queue.cc:260] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.224526 4699 ts_tablet_manager.cc:1398] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.234910 26659 consensus_queue.cc:237] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:06.236032 27064 raft_consensus.cc:1261] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.236212 29216 raft_consensus.cc:1261] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.236362 26595 consensus_peers.cc:485] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 213c37d37f1c4a3c8fbc48deafa9edac. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.236532 4649 consensus_queue.cc:1035] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.236704 3306 consensus_queue.cc:1035] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.238273 3306 raft_consensus.cc:2932] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.238703 29216 raft_consensus.cc:2932] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.240432 27064 raft_consensus.cc:2932] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.241483 26514 catalog_manager.cc:5484] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.245718 4345 consensus_queue.cc:1035] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.248883 3791 consensus_queue.cc:1035] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.264295 26794 consensus_queue.cc:237] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.265447 29213 raft_consensus.cc:1261] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.265761 26730 consensus_peers.cc:485] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3d8471cf7654dbdab51d855ed30be18. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.265765 29080 raft_consensus.cc:1261] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.265867 4564 consensus_queue.cc:1035] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.266144 4734 consensus_queue.cc:1035] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.268311 4734 raft_consensus.cc:2932] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.268699 29079 raft_consensus.cc:2932] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.270262 29213 raft_consensus.cc:2932] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.271559 26515 catalog_manager.cc:5484] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.289420 27064 consensus_queue.cc:237] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.290691 29350 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 01:59:06.290875 26997 consensus_peers.cc:485] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0d96b487a964b90b720d8977a96fc5b. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.291072 4345 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.291196 26659 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:06.291532 4345 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.293465 3791 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.293855 26659 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.295259 29350 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.296644 26514 catalog_manager.cc:5484] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.314431 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:06.320314 29487 consensus_queue.cc:237] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:06.320768 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 4054c8010ebe4c029ee0125581636ea8 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.321046 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4054c8010ebe4c029ee0125581636ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.321028 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 4054c8010ebe4c029ee0125581636ea8 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.321273 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4054c8010ebe4c029ee0125581636ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.321777 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 86bd546634a948069a9f97673dfc188c (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.322057 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86bd546634a948069a9f97673dfc188c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.323374 4771 tablet_bootstrap.cc:492] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:06.323536 29329 tablet_service.cc:1467] Processing CreateTablet for tablet efa94768c376418396d65a10f5d735c4 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.323782 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa94768c376418396d65a10f5d735c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.324342 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 86bd546634a948069a9f97673dfc188c (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.324584 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86bd546634a948069a9f97673dfc188c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.324703 4771 tablet_bootstrap.cc:654] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.325697 4774 tablet_bootstrap.cc:492] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:06.325845 26774 tablet_service.cc:1467] Processing CreateTablet for tablet efa94768c376418396d65a10f5d735c4 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.326050 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa94768c376418396d65a10f5d735c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.326339 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 86bd546634a948069a9f97673dfc188c (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.326581 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86bd546634a948069a9f97673dfc188c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.326877 4697 tablet_bootstrap.cc:492] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:06.327028 4771 tablet_bootstrap.cc:492] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:06.327097 4771 ts_tablet_manager.cc:1367] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:06.327530 4771 raft_consensus.cc:348] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.327623 4771 raft_consensus.cc:374] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.327649 4771 raft_consensus.cc:729] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.327742 4771 consensus_queue.cc:260] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.328016 4771 ts_tablet_manager.cc:1398] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.328840 29454 tablet_service.cc:1467] Processing CreateTablet for tablet efa94768c376418396d65a10f5d735c4 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.329068 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa94768c376418396d65a10f5d735c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.329043 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 4054c8010ebe4c029ee0125581636ea8 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=5d41e2c992114ba690c469a60745bf99]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:06.329283 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4054c8010ebe4c029ee0125581636ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.329753 4697 tablet_bootstrap.cc:654] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.330786 4776 tablet_bootstrap.cc:492] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:06.331660 4776 tablet_bootstrap.cc:654] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.332684 26930 raft_consensus.cc:1261] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.333089 4410 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:06.335249 29419 consensus_peers.cc:485] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a5f8bfeddd1443da789f67949830f99. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.335305 26793 raft_consensus.cc:1261] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.336020 4005 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.337038 4775 tablet_bootstrap.cc:492] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:06.338069 4698 tablet_bootstrap.cc:492] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:06.338341 4775 tablet_bootstrap.cc:654] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.338940 4697 tablet_bootstrap.cc:492] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:06.338984 4698 tablet_bootstrap.cc:654] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.339018 4697 ts_tablet_manager.cc:1367] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 01:59:06.339465 4697 raft_consensus.cc:348] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.339567 4697 raft_consensus.cc:374] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.339596 4697 raft_consensus.cc:729] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.339700 4697 consensus_queue.cc:260] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.339955 4697 ts_tablet_manager.cc:1398] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:06.340085 4774 tablet_bootstrap.cc:654] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.340849 4772 tablet_bootstrap.cc:492] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:06.341147 4698 tablet_bootstrap.cc:492] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:06.341217 4698 ts_tablet_manager.cc:1367] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:06.341619 4698 raft_consensus.cc:348] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.341711 4698 raft_consensus.cc:374] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.341738 4698 raft_consensus.cc:729] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.341830 4772 tablet_bootstrap.cc:654] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.341881 4698 consensus_queue.cc:260] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.342164 4698 ts_tablet_manager.cc:1398] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.344903 4774 tablet_bootstrap.cc:492] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:06.345033 4774 ts_tablet_manager.cc:1367] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 01:59:06.345443 4774 raft_consensus.cc:348] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.345713 4774 raft_consensus.cc:374] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.345810 4774 raft_consensus.cc:729] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.345937 4774 consensus_queue.cc:260] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.346916 4776 tablet_bootstrap.cc:492] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:06.347009 4774 ts_tablet_manager.cc:1398] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:06.347045 4776 ts_tablet_manager.cc:1367] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 01:59:06.347548 4776 raft_consensus.cc:348] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.347680 4776 raft_consensus.cc:374] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.347731 4776 raft_consensus.cc:729] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.347857 4776 consensus_queue.cc:260] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.348152 4776 ts_tablet_manager.cc:1398] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.347381 4775 tablet_bootstrap.cc:492] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:06.348362 4775 ts_tablet_manager.cc:1367] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:06.348752 4775 raft_consensus.cc:348] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.348838 4775 raft_consensus.cc:374] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.348866 4775 raft_consensus.cc:729] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.348980 4775 consensus_queue.cc:260] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.349258 4775 ts_tablet_manager.cc:1398] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.349407 4775 tablet_bootstrap.cc:492] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:06.350425 2712 raft_consensus.cc:482] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:06.350518 2712 raft_consensus.cc:504] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.350874 2712 leader_election.cc:288] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.351128 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054c8010ebe4c029ee0125581636ea8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:06.351261 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054c8010ebe4c029ee0125581636ea8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:06.351365 27064 raft_consensus.cc:2454] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
W20240502 01:59:06.351455 29015 leader_election.cc:341] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:06.351624 29016 leader_election.cc:302] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters: b290c1bf893444078ed721f25995d124
I20240502 01:59:06.351835 2712 raft_consensus.cc:2781] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:06.351883 2712 raft_consensus.cc:482] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:06.351917 2712 raft_consensus.cc:3037] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.352840 2712 raft_consensus.cc:504] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.353165 2712 leader_election.cc:288] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.353523 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054c8010ebe4c029ee0125581636ea8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:06.353431 4410 raft_consensus.cc:2932] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:06.353617 27064 raft_consensus.cc:3037] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.353942 4775 tablet_bootstrap.cc:654] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.354185 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054c8010ebe4c029ee0125581636ea8" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
W20240502 01:59:06.354542 29015 leader_election.cc:341] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:06.354555 27064 raft_consensus.cc:2454] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:06.354838 29016 leader_election.cc:302] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters: b290c1bf893444078ed721f25995d124
I20240502 01:59:06.355048 2712 raft_consensus.cc:2781] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:06.355118 2712 raft_consensus.cc:686] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:06.355158 26930 raft_consensus.cc:2932] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:06.355255 4772 tablet_bootstrap.cc:492] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:06.355222 2712 consensus_queue.cc:237] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.355336 4772 ts_tablet_manager.cc:1367] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:59:06.355757 4772 raft_consensus.cc:348] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.355851 4772 raft_consensus.cc:374] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.355880 4772 raft_consensus.cc:729] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.355979 4772 consensus_queue.cc:260] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.356230 4772 ts_tablet_manager.cc:1398] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.356381 4772 tablet_bootstrap.cc:492] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:06.357117 4772 tablet_bootstrap.cc:654] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:06.357978 26514 catalog_manager.cc:5484] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.358981 26515 catalog_manager.cc:5484] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:06.359927 26794 raft_consensus.cc:2932] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:06.362567 4005 raft_consensus.cc:482] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:06.362654 4775 tablet_bootstrap.cc:492] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:06.362640 4005 raft_consensus.cc:504] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.362733 4775 ts_tablet_manager.cc:1367] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:59:06.362959 4005 leader_election.cc:288] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:06.363188 4775 raft_consensus.cc:348] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.363276 4775 raft_consensus.cc:374] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.363306 4775 raft_consensus.cc:729] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.363291 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:06.363391 29213 raft_consensus.cc:2454] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:06.363664 29421 leader_election.cc:302] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:06.363943 4005 raft_consensus.cc:2781] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:06.364010 4005 raft_consensus.cc:482] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:06.364048 4005 raft_consensus.cc:3037] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.364248 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:06.364261 4775 consensus_queue.cc:260] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.364346 26930 raft_consensus.cc:2454] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:06.364527 4775 ts_tablet_manager.cc:1398] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:06.364940 4005 raft_consensus.cc:504] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.365273 4005 leader_election.cc:288] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:06.365552 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:06.365659 26930 raft_consensus.cc:3037] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.365694 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:06.365789 29213 raft_consensus.cc:3037] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.366514 26930 raft_consensus.cc:2454] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:06.366755 29422 leader_election.cc:302] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:06.366786 29213 raft_consensus.cc:2454] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:06.366922 4005 raft_consensus.cc:2781] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:06.367017 4005 raft_consensus.cc:686] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:06.367156 4005 consensus_queue.cc:237] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.370551 4772 tablet_bootstrap.cc:492] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:06.370636 4772 ts_tablet_manager.cc:1367] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:06.371038 4772 raft_consensus.cc:348] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.371131 4772 raft_consensus.cc:374] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:06.371160 4772 raft_consensus.cc:729] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.371265 4772 consensus_queue.cc:260] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.371573 4772 ts_tablet_manager.cc:1398] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.372359 26507 catalog_manager.cc:5484] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.390100 27064 consensus_queue.cc:237] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
W20240502 01:59:06.391408 26998 consensus_peers.cc:485] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b546a99821624f0588a4c0f59a5ec5a3. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.391408 26659 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.391415 26930 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.391738 4801 ts_tablet_manager.cc:902] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:06.391762 4755 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.391943 4345 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.392244 4801 tablet_copy_client.cc:308] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.198:40103
I20240502 01:59:06.394320 4345 raft_consensus.cc:2932] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.394696 26930 raft_consensus.cc:2932] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.396927 26507 catalog_manager.cc:5484] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.397133 26659 raft_consensus.cc:2932] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.398988 29221 tablet_copy_service.cc:140] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Received BeginTabletCopySession request for tablet 499d2ca7bffc42748591db5280f829f2 from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:43975)
I20240502 01:59:06.399104 29221 tablet_copy_service.cc:161] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Beginning new tablet copy session on tablet 499d2ca7bffc42748591db5280f829f2 from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:43975: session id = b290c1bf893444078ed721f25995d124-499d2ca7bffc42748591db5280f829f2
I20240502 01:59:06.400502 29221 tablet_copy_source_session.cc:205] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.402158 4801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499d2ca7bffc42748591db5280f829f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.405145 4801 tablet_copy_client.cc:783] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.405356 4801 tablet_copy_client.cc:647] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.406659 4801 tablet_copy_client.cc:523] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.407657 4801 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:06.410810 4801 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.411070 4801 tablet_bootstrap.cc:492] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:59:06.411221 4801 ts_tablet_manager.cc:1367] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20240502 01:59:06.411695 4801 raft_consensus.cc:348] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.411857 4801 raft_consensus.cc:729] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.412045 4801 consensus_queue.cc:260] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.412381 4801 ts_tablet_manager.cc:1398] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:06.412943 29221 tablet_copy_service.cc:342] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Request end of tablet copy session b290c1bf893444078ed721f25995d124-499d2ca7bffc42748591db5280f829f2 received from {username='slave'} at 127.19.43.194:43975
I20240502 01:59:06.413017 29221 tablet_copy_service.cc:433] P 0d83fdd1e6e04cf5a5602994d1dbc0df: ending tablet copy session b290c1bf893444078ed721f25995d124-499d2ca7bffc42748591db5280f829f2 on tablet 499d2ca7bffc42748591db5280f829f2 with peer b290c1bf893444078ed721f25995d124
I20240502 01:59:06.423074 29350 consensus_queue.cc:237] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.424383 26659 raft_consensus.cc:1261] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.424443 29079 raft_consensus.cc:1261] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.424482 29286 consensus_peers.cc:485] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f701a3437f84a84b6e3290266d24c94. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.424746 4551 consensus_queue.cc:1035] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.424942 4573 consensus_queue.cc:1035] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.426525 4573 raft_consensus.cc:2932] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.426895 29079 raft_consensus.cc:2932] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.428180 26659 raft_consensus.cc:2932] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.429536 26513 catalog_manager.cc:5484] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:06.430318 3297 raft_consensus.cc:482] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:06.430402 3297 raft_consensus.cc:504] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.430827 3297 leader_election.cc:288] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:06.431069 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7317c932fdba4235816aff6d5c482878" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:06.431193 27064 raft_consensus.cc:2454] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:06.431630 29150 leader_election.cc:302] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:06.431960 4527 raft_consensus.cc:2781] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:06.432018 4527 raft_consensus.cc:482] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:06.432053 4527 raft_consensus.cc:3037] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.433032 4527 raft_consensus.cc:504] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.433192 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7317c932fdba4235816aff6d5c482878" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:06.433307 26659 raft_consensus.cc:2454] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:06.433373 4527 leader_election.cc:288] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:06.433874 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7317c932fdba4235816aff6d5c482878" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:06.433969 26659 raft_consensus.cc:3037] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.434238 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7317c932fdba4235816aff6d5c482878" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:06.434351 27064 raft_consensus.cc:3037] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.434764 26659 raft_consensus.cc:2454] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:06.435007 29149 leader_election.cc:302] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:06.435367 27064 raft_consensus.cc:2454] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:06.435683 3297 raft_consensus.cc:2781] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:06.435776 3297 raft_consensus.cc:686] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:06.435897 3297 consensus_queue.cc:237] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.437877 26514 catalog_manager.cc:5484] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:06.457469 26659 consensus_queue.cc:237] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.459030 29079 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.459146 26794 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.459654 26594 consensus_peers.cc:485] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d216b8ccd8ea43df85c1013cdaf36564. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.459717 4649 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.460289 4649 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.462013 4646 raft_consensus.cc:2932] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.462479 26794 raft_consensus.cc:2932] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.464355 29079 raft_consensus.cc:2932] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.466902 26514 catalog_manager.cc:5484] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index -1 to 2, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.482921 26930 consensus_queue.cc:237] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:06.484244 29213 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:06.484355 26793 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 01:59:06.484411 26862 consensus_peers.cc:485] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 626774246b9445bc838427358e08f2cd. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.484715 4669 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.484882 3124 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.486580 4669 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.486999 26793 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.489775 26507 catalog_manager.cc:5484] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.490203 29213 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:06.493942 4876 ts_tablet_manager.cc:902] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: Initiating tablet copy from peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:06.494464 4876 tablet_copy_client.cc:308] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.200:36259
I20240502 01:59:06.494843 29512 tablet_copy_service.cc:140] P 7c693d0933c64d9babc07f404df45cd1: Received BeginTabletCopySession request for tablet 8209cefcc49042b18fdd97d845d167e2 from peer 68e5b1eeb04448fea818a14991eab87c ({username='slave'} at 127.19.43.196:55395)
I20240502 01:59:06.494959 29512 tablet_copy_service.cc:161] P 7c693d0933c64d9babc07f404df45cd1: Beginning new tablet copy session on tablet 8209cefcc49042b18fdd97d845d167e2 from peer 68e5b1eeb04448fea818a14991eab87c at {username='slave'} at 127.19.43.196:55395: session id = 68e5b1eeb04448fea818a14991eab87c-8209cefcc49042b18fdd97d845d167e2
I20240502 01:59:06.495769 29512 tablet_copy_source_session.cc:205] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.496600 4876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8209cefcc49042b18fdd97d845d167e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.499325 4876 tablet_copy_client.cc:783] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.499477 4876 tablet_copy_client.cc:647] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.500485 4876 tablet_copy_client.cc:523] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.501590 4876 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:06.505255 4876 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.505497 4876 tablet_bootstrap.cc:492] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap complete.
I20240502 01:59:06.505654 4876 ts_tablet_manager.cc:1367] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240502 01:59:06.506203 4876 raft_consensus.cc:348] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.506296 4876 raft_consensus.cc:729] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.506392 4876 consensus_queue.cc:260] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.506644 4876 ts_tablet_manager.cc:1398] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:59:06.507027 29512 tablet_copy_service.cc:342] P 7c693d0933c64d9babc07f404df45cd1: Request end of tablet copy session 68e5b1eeb04448fea818a14991eab87c-8209cefcc49042b18fdd97d845d167e2 received from {username='slave'} at 127.19.43.196:55395
I20240502 01:59:06.507081 29512 tablet_copy_service.cc:433] P 7c693d0933c64d9babc07f404df45cd1: ending tablet copy session 68e5b1eeb04448fea818a14991eab87c-8209cefcc49042b18fdd97d845d167e2 on tablet 8209cefcc49042b18fdd97d845d167e2 with peer 68e5b1eeb04448fea818a14991eab87c
I20240502 01:59:06.509721 29489 consensus_queue.cc:237] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.511191 29079 raft_consensus.cc:1261] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.511232 29350 raft_consensus.cc:1261] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:06.511572 29422 consensus_peers.cc:485] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ca8563ff68c4fa590c17ef58e67afa0. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.511737 4789 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.511924 4005 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:06.513669 4789 raft_consensus.cc:2932] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.514423 29079 raft_consensus.cc:2932] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.515362 29350 raft_consensus.cc:2932] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.516443 26513 catalog_manager.cc:5484] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:06.522226 4801 ts_tablet_manager.cc:902] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:06.522662 4801 tablet_copy_client.cc:308] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.199:44303
I20240502 01:59:06.523102 29369 tablet_copy_service.cc:140] P 468dd27456c64d67b22e8a58af93d93d: Received BeginTabletCopySession request for tablet 273b6d65e3df42ffb072e2129e832a92 from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:58085)
I20240502 01:59:06.523190 29369 tablet_copy_service.cc:161] P 468dd27456c64d67b22e8a58af93d93d: Beginning new tablet copy session on tablet 273b6d65e3df42ffb072e2129e832a92 from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:58085: session id = b290c1bf893444078ed721f25995d124-273b6d65e3df42ffb072e2129e832a92
I20240502 01:59:06.523964 29369 tablet_copy_source_session.cc:205] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.524763 4801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 273b6d65e3df42ffb072e2129e832a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.526626 4801 tablet_copy_client.cc:783] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.526757 4801 tablet_copy_client.cc:647] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.527815 4801 tablet_copy_client.cc:523] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.528959 4801 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:06.531841 4801 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.532091 4801 tablet_bootstrap.cc:492] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:59:06.532300 4801 ts_tablet_manager.cc:1367] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.001s
I20240502 01:59:06.532742 4801 raft_consensus.cc:348] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.532871 4801 raft_consensus.cc:729] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.533010 4801 consensus_queue.cc:260] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
W20240502 01:59:06.533449 26978 tablet.cc:2369] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:59:06.535089 4801 ts_tablet_manager.cc:1398] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:06.535959 29369 tablet_copy_service.cc:342] P 468dd27456c64d67b22e8a58af93d93d: Request end of tablet copy session b290c1bf893444078ed721f25995d124-273b6d65e3df42ffb072e2129e832a92 received from {username='slave'} at 127.19.43.194:58085
I20240502 01:59:06.536031 29369 tablet_copy_service.cc:433] P 468dd27456c64d67b22e8a58af93d93d: ending tablet copy session b290c1bf893444078ed721f25995d124-273b6d65e3df42ffb072e2129e832a92 on tablet 273b6d65e3df42ffb072e2129e832a92 with peer b290c1bf893444078ed721f25995d124
I20240502 01:59:06.538014 27064 consensus_queue.cc:237] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.539511 26930 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:06.539800 29079 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 01:59:06.539863 26997 consensus_peers.cc:485] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c -> Peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): Couldn't send request to peer 0d83fdd1e6e04cf5a5602994d1dbc0df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1950ace91824415b9bf3d1657bad72f. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:06.539880 4451 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.540194 3791 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:06.542129 4451 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.542513 29079 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.545604 26507 catalog_manager.cc:5484] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.546795 4918 ts_tablet_manager.cc:902] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:06.546905 26930 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:06.547449 4918 tablet_copy_client.cc:308] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:59:06.547868 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet e1cd4681de8645ff92b479ccdd3eac37 from peer f3d18a019013487ab801599cfb80e4b4 ({username='slave'} at 127.19.43.197:39103)
I20240502 01:59:06.547969 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet e1cd4681de8645ff92b479ccdd3eac37 from peer f3d18a019013487ab801599cfb80e4b4 at {username='slave'} at 127.19.43.197:39103: session id = f3d18a019013487ab801599cfb80e4b4-e1cd4681de8645ff92b479ccdd3eac37
I20240502 01:59:06.548970 26682 tablet_copy_source_session.cc:205] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.549831 4918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1cd4681de8645ff92b479ccdd3eac37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.552474 4918 tablet_copy_client.cc:783] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.552664 4918 tablet_copy_client.cc:647] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.559352 4918 tablet_copy_client.cc:523] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.560484 4918 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:06.570140 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:06.582720 4734 raft_consensus.cc:482] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:06.582808 4734 raft_consensus.cc:504] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.583158 4734 leader_election.cc:288] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:06.583442 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa94768c376418396d65a10f5d735c4" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:06.583561 29489 raft_consensus.cc:2454] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:06.583541 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa94768c376418396d65a10f5d735c4" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:06.583644 29350 raft_consensus.cc:2454] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:06.583959 26727 leader_election.cc:302] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:06.584033 26659 raft_consensus.cc:1261] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.584206 4734 raft_consensus.cc:2781] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:06.584267 4734 raft_consensus.cc:482] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:06.584321 4734 raft_consensus.cc:3037] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.584743 4526 consensus_queue.cc:1035] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.585304 4734 raft_consensus.cc:504] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.585631 4734 leader_election.cc:288] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:06.585649 4918 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.585899 4918 tablet_bootstrap.cc:492] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: Bootstrap complete.
I20240502 01:59:06.585896 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa94768c376418396d65a10f5d735c4" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:06.586005 29350 raft_consensus.cc:3037] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.586076 4918 ts_tablet_manager.cc:1367] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240502 01:59:06.586210 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa94768c376418396d65a10f5d735c4" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:06.586318 29489 raft_consensus.cc:3037] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.586472 4918 raft_consensus.cc:348] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:06.586570 4918 raft_consensus.cc:729] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.586702 4918 consensus_queue.cc:260] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:06.587009 29350 raft_consensus.cc:2454] T efa94768c376418396d65a10f5d735c4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:06.587190 29489 raft_consensus.cc:2454] T efa94768c376418396d65a10f5d735c4 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:06.587743 4918 ts_tablet_manager.cc:1398] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:06.588126 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session f3d18a019013487ab801599cfb80e4b4-e1cd4681de8645ff92b479ccdd3eac37 received from {username='slave'} at 127.19.43.197:39103
I20240502 01:59:06.588194 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session f3d18a019013487ab801599cfb80e4b4-e1cd4681de8645ff92b479ccdd3eac37 on tablet e1cd4681de8645ff92b479ccdd3eac37 with peer f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:06.588335 27064 raft_consensus.cc:1261] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:06.588848 26515 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]). Expected version 1 got 0
I20240502 01:59:06.588845 4526 consensus_queue.cc:1035] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.590128 26727 leader_election.cc:302] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:06.590689 4734 raft_consensus.cc:2781] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:06.590768 4734 raft_consensus.cc:686] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:06.590829 4689 mvcc.cc:205] Tried to move back new op lower bound from 7023063640398344192 to 7023063639802707968. Current Snapshot: MvccSnapshot[applied={T|T < 7023063640398344192}]
I20240502 01:59:06.590876 4734 consensus_queue.cc:237] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.590950 4944 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:06.591223 4944 tablet.cc:1620] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:06.591738 26515 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]). Expected version 1 got 0
I20240502 01:59:06.594208 26515 catalog_manager.cc:5484] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:06.594834 26513 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]). Expected version 1 got 0
I20240502 01:59:06.595212 4362 mvcc.cc:205] Tried to move back new op lower bound from 7023063640398344192 to 7023063639802707968. Current Snapshot: MvccSnapshot[applied={T|T < 7023063640398344192}]
I20240502 01:59:06.598421 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3] alter complete (version 1)
I20240502 01:59:06.598604 26514 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3]). Expected version 1 got 0
I20240502 01:59:06.598953 4945 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:06.599334 4945 tablet.cc:1620] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:06.600517 4947 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:06.600798 4947 tablet.cc:1620] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:06.603094 4948 ts_tablet_manager.cc:902] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: Initiating tablet copy from peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:06.604305 4948 tablet_copy_client.cc:308] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.199:44303
I20240502 01:59:06.604725 29369 tablet_copy_service.cc:140] P 468dd27456c64d67b22e8a58af93d93d: Received BeginTabletCopySession request for tablet 78609c45dc0c4e1b838bb737ebf7e034 from peer 998b57e2badc4d779c586c6579149114 ({username='slave'} at 127.19.43.193:55349)
I20240502 01:59:06.604817 29369 tablet_copy_service.cc:161] P 468dd27456c64d67b22e8a58af93d93d: Beginning new tablet copy session on tablet 78609c45dc0c4e1b838bb737ebf7e034 from peer 998b57e2badc4d779c586c6579149114 at {username='slave'} at 127.19.43.193:55349: session id = 998b57e2badc4d779c586c6579149114-78609c45dc0c4e1b838bb737ebf7e034
I20240502 01:59:06.605727 29369 tablet_copy_source_session.cc:205] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.605983 4936 ts_tablet_manager.cc:902] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:06.606459 4936 tablet_copy_client.cc:308] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:59:06.606546 4948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78609c45dc0c4e1b838bb737ebf7e034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.606726 4937 ts_tablet_manager.cc:902] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:06.606890 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet 8afd4c5362464c4cba43d5ef28b50c29 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:49803)
I20240502 01:59:06.606976 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet 8afd4c5362464c4cba43d5ef28b50c29 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:49803: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-8afd4c5362464c4cba43d5ef28b50c29
I20240502 01:59:06.607313 4937 tablet_copy_client.cc:308] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.199:44303
I20240502 01:59:06.607813 29102 tablet_copy_source_session.cc:205] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.608294 29369 tablet_copy_service.cc:140] P 468dd27456c64d67b22e8a58af93d93d: Received BeginTabletCopySession request for tablet 089d9dc33a594712ac04ad28a0cd37b1 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:43081)
I20240502 01:59:06.608372 29369 tablet_copy_service.cc:161] P 468dd27456c64d67b22e8a58af93d93d: Beginning new tablet copy session on tablet 089d9dc33a594712ac04ad28a0cd37b1 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:43081: session id = 842c1b011b5d4c77bfd9b9591111502e-089d9dc33a594712ac04ad28a0cd37b1
I20240502 01:59:06.608619 4936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8afd4c5362464c4cba43d5ef28b50c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.608633 4948 tablet_copy_client.cc:783] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.608753 4948 tablet_copy_client.cc:647] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.609102 29369 tablet_copy_source_session.cc:205] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.610718 4936 tablet_copy_client.cc:783] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.610864 4936 tablet_copy_client.cc:647] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.610908 4937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089d9dc33a594712ac04ad28a0cd37b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.611536 4948 tablet_copy_client.cc:523] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.612586 4948 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:06.613049 4937 tablet_copy_client.cc:783] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.613184 4937 tablet_copy_client.cc:647] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.615500 4936 tablet_copy_client.cc:523] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.616479 4937 tablet_copy_client.cc:523] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.617262 4936 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:06.617655 4937 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:06.617820 4948 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.618108 4948 tablet_bootstrap.cc:492] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: Bootstrap complete.
I20240502 01:59:06.618257 4948 ts_tablet_manager.cc:1367] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20240502 01:59:06.618670 4948 raft_consensus.cc:348] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.618789 4948 raft_consensus.cc:729] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.618934 4948 consensus_queue.cc:260] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:06.619414 4948 ts_tablet_manager.cc:1398] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:06.620404 4801 ts_tablet_manager.cc:902] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.620452 29369 tablet_copy_service.cc:342] P 468dd27456c64d67b22e8a58af93d93d: Request end of tablet copy session 998b57e2badc4d779c586c6579149114-78609c45dc0c4e1b838bb737ebf7e034 received from {username='slave'} at 127.19.43.193:55349
I20240502 01:59:06.620513 29369 tablet_copy_service.cc:433] P 468dd27456c64d67b22e8a58af93d93d: ending tablet copy session 998b57e2badc4d779c586c6579149114-78609c45dc0c4e1b838bb737ebf7e034 on tablet 78609c45dc0c4e1b838bb737ebf7e034 with peer 998b57e2badc4d779c586c6579149114
I20240502 01:59:06.620807 4801 tablet_copy_client.cc:308] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:59:06.621296 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet ab7a9b176213400f990c4d5df0d7567f from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:40063)
I20240502 01:59:06.621387 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet ab7a9b176213400f990c4d5df0d7567f from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:40063: session id = b290c1bf893444078ed721f25995d124-ab7a9b176213400f990c4d5df0d7567f
I20240502 01:59:06.622196 4937 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.622311 27083 tablet_copy_source_session.cc:205] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.622457 4937 tablet_bootstrap.cc:492] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:06.622593 4937 ts_tablet_manager.cc:1367] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 01:59:06.623004 4937 raft_consensus.cc:348] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.623108 4937 raft_consensus.cc:729] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.623245 4937 consensus_queue.cc:260] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.623776 4937 ts_tablet_manager.cc:1398] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.623975 4936 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.624217 4936 tablet_bootstrap.cc:492] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:59:06.624220 29369 tablet_copy_service.cc:342] P 468dd27456c64d67b22e8a58af93d93d: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-089d9dc33a594712ac04ad28a0cd37b1 received from {username='slave'} at 127.19.43.195:43081
I20240502 01:59:06.624285 29369 tablet_copy_service.cc:433] P 468dd27456c64d67b22e8a58af93d93d: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-089d9dc33a594712ac04ad28a0cd37b1 on tablet 089d9dc33a594712ac04ad28a0cd37b1 with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:06.624351 4936 ts_tablet_manager.cc:1367] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 01:59:06.624737 4936 raft_consensus.cc:348] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.624838 4936 raft_consensus.cc:729] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.624953 4936 consensus_queue.cc:260] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.625200 4936 ts_tablet_manager.cc:1398] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.625677 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-8afd4c5362464c4cba43d5ef28b50c29 received from {username='slave'} at 127.19.43.198:49803
I20240502 01:59:06.625743 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-8afd4c5362464c4cba43d5ef28b50c29 on tablet 8afd4c5362464c4cba43d5ef28b50c29 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:59:06.625941 4801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab7a9b176213400f990c4d5df0d7567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.628178 4801 tablet_copy_client.cc:783] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.628312 4801 tablet_copy_client.cc:647] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.632036 4801 tablet_copy_client.cc:523] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.633189 4801 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:06.636538 4801 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.636768 4801 tablet_bootstrap.cc:492] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:59:06.636885 4801 ts_tablet_manager.cc:1367] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:59:06.637228 4801 raft_consensus.cc:348] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.637347 4801 raft_consensus.cc:729] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.637439 4801 consensus_queue.cc:260] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:06.637665 4801 ts_tablet_manager.cc:1398] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.638131 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session b290c1bf893444078ed721f25995d124-ab7a9b176213400f990c4d5df0d7567f received from {username='slave'} at 127.19.43.194:40063
I20240502 01:59:06.638206 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session b290c1bf893444078ed721f25995d124-ab7a9b176213400f990c4d5df0d7567f on tablet ab7a9b176213400f990c4d5df0d7567f with peer b290c1bf893444078ed721f25995d124
I20240502 01:59:06.670199 4936 ts_tablet_manager.cc:902] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.670607 4936 tablet_copy_client.cc:308] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:59:06.671017 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet a0d96b487a964b90b720d8977a96fc5b from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:58871)
I20240502 01:59:06.671139 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet a0d96b487a964b90b720d8977a96fc5b from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:58871: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-a0d96b487a964b90b720d8977a96fc5b
I20240502 01:59:06.672061 27083 tablet_copy_source_session.cc:205] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.672848 4936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d96b487a964b90b720d8977a96fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.675037 4936 tablet_copy_client.cc:783] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.675175 4936 tablet_copy_client.cc:647] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.676604 3993 raft_consensus.cc:482] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:06.676697 3993 raft_consensus.cc:504] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:06.677129 3993 leader_election.cc:288] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.677695 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054c8010ebe4c029ee0125581636ea8" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:06.677841 27064 raft_consensus.cc:2379] T 4054c8010ebe4c029ee0125581636ea8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b290c1bf893444078ed721f25995d124 in current term 1: Already voted for candidate f3d18a019013487ab801599cfb80e4b4 in this term.
I20240502 01:59:06.678375 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4054c8010ebe4c029ee0125581636ea8" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:06.679549 26730 leader_election.cc:302] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b290c1bf893444078ed721f25995d124; no voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:06.679706 4936 tablet_copy_client.cc:523] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.679867 3993 raft_consensus.cc:3037] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:06.680967 3993 raft_consensus.cc:2726] T 4054c8010ebe4c029ee0125581636ea8 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 01:59:06.681752 4936 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:06.682255 4999 ts_tablet_manager.cc:902] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:06.682825 4999 tablet_copy_client.cc:308] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:59:06.683324 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet 213c37d37f1c4a3c8fbc48deafa9edac from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:59383)
I20240502 01:59:06.683424 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet 213c37d37f1c4a3c8fbc48deafa9edac from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:59383: session id = 7c693d0933c64d9babc07f404df45cd1-213c37d37f1c4a3c8fbc48deafa9edac
I20240502 01:59:06.684439 26682 tablet_copy_source_session.cc:205] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.685391 4999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213c37d37f1c4a3c8fbc48deafa9edac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.687609 4999 tablet_copy_client.cc:783] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.687738 4999 tablet_copy_client.cc:647] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.688828 4999 tablet_copy_client.cc:523] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.692847 4999 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:06.696959 4999 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.697198 4999 tablet_bootstrap.cc:492] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:59:06.697346 4999 ts_tablet_manager.cc:1367] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20240502 01:59:06.697778 4999 raft_consensus.cc:348] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:06.697893 4999 raft_consensus.cc:729] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.698001 4999 consensus_queue.cc:260] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:06.698462 4999 ts_tablet_manager.cc:1398] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:59:06.699532 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-213c37d37f1c4a3c8fbc48deafa9edac received from {username='slave'} at 127.19.43.200:59383
I20240502 01:59:06.699599 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-213c37d37f1c4a3c8fbc48deafa9edac on tablet 213c37d37f1c4a3c8fbc48deafa9edac with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:06.700393 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:06.704679 5005 ts_tablet_manager.cc:902] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:06.705296 5005 tablet_copy_client.cc:308] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:59:06.706874 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 5515469fdf494900ae20f20fdfca587c from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:51525)
I20240502 01:59:06.706970 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 5515469fdf494900ae20f20fdfca587c from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:51525: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-5515469fdf494900ae20f20fdfca587c
I20240502 01:59:06.707207 4944 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:06.707484 4944 tablet.cc:1620] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:06.707989 26950 tablet_copy_source_session.cc:205] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.711262 5005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5515469fdf494900ae20f20fdfca587c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.713608 4945 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:06.713876 4945 tablet.cc:1620] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:06.714872 5005 tablet_copy_client.cc:783] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.715004 5005 tablet_copy_client.cc:647] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.715974 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=329f83c2a6144f59993e545d2b3209e3] alter complete (version 2)
I20240502 01:59:06.715988 4947 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:06.716267 4947 tablet.cc:1620] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:06.726569 5005 tablet_copy_client.cc:523] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.727710 5005 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:06.733323 29195 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:06.737977 26774 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:06.739323 29329 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:06.742264 2712 consensus_queue.cc:1035] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.742894 4936 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.745000 4936 tablet_bootstrap.cc:492] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:59:06.745194 4936 ts_tablet_manager.cc:1367] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.063s user 0.024s sys 0.002s
I20240502 01:59:06.745550 4936 raft_consensus.cc:348] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.745652 4936 raft_consensus.cc:729] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.745744 4936 consensus_queue.cc:260] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.746165 4936 ts_tablet_manager.cc:1398] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:06.746587 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-a0d96b487a964b90b720d8977a96fc5b received from {username='slave'} at 127.19.43.198:58871
I20240502 01:59:06.746647 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-a0d96b487a964b90b720d8977a96fc5b on tablet a0d96b487a964b90b720d8977a96fc5b with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:59:06.747381 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:06.752293 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:06.756949 26640 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:06.759016 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:06.777320 4592 consensus_queue.cc:1035] T 4054c8010ebe4c029ee0125581636ea8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.792793 4779 consensus_queue.cc:1035] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.804369 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:06.806169 5005 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.806440 5005 tablet_bootstrap.cc:492] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:59:06.806648 5005 ts_tablet_manager.cc:1367] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.079s user 0.026s sys 0.000s
I20240502 01:59:06.807060 5005 raft_consensus.cc:348] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.807189 5005 raft_consensus.cc:729] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.807330 5005 consensus_queue.cc:260] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.807638 5005 ts_tablet_manager.cc:1398] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.814112 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-5515469fdf494900ae20f20fdfca587c received from {username='slave'} at 127.19.43.198:51525
I20240502 01:59:06.814191 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-5515469fdf494900ae20f20fdfca587c on tablet 5515469fdf494900ae20f20fdfca587c with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:59:06.816182 4005 consensus_queue.cc:1035] T 86bd546634a948069a9f97673dfc188c P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:06.819907 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:06.825352 4944 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:06.825932 4945 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:06.826162 4947 tablet.cc:1722] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:06.831658 4937 ts_tablet_manager.cc:902] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:06.832137 4937 tablet_copy_client.cc:308] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:06.832732 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=329f83c2a6144f59993e545d2b3209e3] alter complete (version 3)
I20240502 01:59:06.833817 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet d3d8471cf7654dbdab51d855ed30be18 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:36821)
I20240502 01:59:06.833912 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet d3d8471cf7654dbdab51d855ed30be18 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:36821: session id = 842c1b011b5d4c77bfd9b9591111502e-d3d8471cf7654dbdab51d855ed30be18
I20240502 01:59:06.834506 5005 ts_tablet_manager.cc:902] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:06.834825 26815 tablet_copy_source_session.cc:205] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.834914 5005 tablet_copy_client.cc:308] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.193:32929
I20240502 01:59:06.835424 26682 tablet_copy_service.cc:140] P 998b57e2badc4d779c586c6579149114: Received BeginTabletCopySession request for tablet d216b8ccd8ea43df85c1013cdaf36564 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:45005)
I20240502 01:59:06.835508 26682 tablet_copy_service.cc:161] P 998b57e2badc4d779c586c6579149114: Beginning new tablet copy session on tablet d216b8ccd8ea43df85c1013cdaf36564 from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:45005: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-d216b8ccd8ea43df85c1013cdaf36564
I20240502 01:59:06.835631 4937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d8471cf7654dbdab51d855ed30be18. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.836344 26682 tablet_copy_source_session.cc:205] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.837092 5005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d216b8ccd8ea43df85c1013cdaf36564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.837731 4937 tablet_copy_client.cc:783] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.837868 4937 tablet_copy_client.cc:647] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.839365 5005 tablet_copy_client.cc:783] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.839551 5005 tablet_copy_client.cc:647] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.841043 4937 tablet_copy_client.cc:523] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.841933 4937 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:06.842957 5005 tablet_copy_client.cc:523] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.844014 5005 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:06.846104 4937 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.846307 4937 tablet_bootstrap.cc:492] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:06.846426 4937 ts_tablet_manager.cc:1367] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 01:59:06.846796 4937 raft_consensus.cc:348] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.846887 4937 raft_consensus.cc:729] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.846995 4937 consensus_queue.cc:260] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.847257 4937 ts_tablet_manager.cc:1398] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.847970 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-d3d8471cf7654dbdab51d855ed30be18 received from {username='slave'} at 127.19.43.195:36821
I20240502 01:59:06.848029 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-d3d8471cf7654dbdab51d855ed30be18 on tablet d3d8471cf7654dbdab51d855ed30be18 with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:06.855291 5005 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.855511 5005 tablet_bootstrap.cc:492] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:59:06.855645 5005 ts_tablet_manager.cc:1367] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 01:59:06.856019 5005 raft_consensus.cc:348] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.856113 5005 raft_consensus.cc:729] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.856228 5005 consensus_queue.cc:260] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:06.856463 5005 ts_tablet_manager.cc:1398] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.856911 26682 tablet_copy_service.cc:342] P 998b57e2badc4d779c586c6579149114: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-d216b8ccd8ea43df85c1013cdaf36564 received from {username='slave'} at 127.19.43.198:45005
I20240502 01:59:06.856998 26682 tablet_copy_service.cc:433] P 998b57e2badc4d779c586c6579149114: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-d216b8ccd8ea43df85c1013cdaf36564 on tablet d216b8ccd8ea43df85c1013cdaf36564 with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:59:06.872252 4999 ts_tablet_manager.cc:902] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.872630 4999 tablet_copy_client.cc:308] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:59:06.873039 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet b546a99821624f0588a4c0f59a5ec5a3 from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:53349)
I20240502 01:59:06.873122 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet b546a99821624f0588a4c0f59a5ec5a3 from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:53349: session id = 7c693d0933c64d9babc07f404df45cd1-b546a99821624f0588a4c0f59a5ec5a3
I20240502 01:59:06.873868 27083 tablet_copy_source_session.cc:205] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.874800 4999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b546a99821624f0588a4c0f59a5ec5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.876612 4999 tablet_copy_client.cc:783] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.876766 4999 tablet_copy_client.cc:647] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.877743 4999 tablet_copy_client.cc:523] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.878973 4999 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:06.879985 4937 ts_tablet_manager.cc:902] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:06.880419 4937 tablet_copy_client.cc:308] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.199:44303
I20240502 01:59:06.880846 29369 tablet_copy_service.cc:140] P 468dd27456c64d67b22e8a58af93d93d: Received BeginTabletCopySession request for tablet 7f701a3437f84a84b6e3290266d24c94 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:43081)
I20240502 01:59:06.880955 29369 tablet_copy_service.cc:161] P 468dd27456c64d67b22e8a58af93d93d: Beginning new tablet copy session on tablet 7f701a3437f84a84b6e3290266d24c94 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:43081: session id = 842c1b011b5d4c77bfd9b9591111502e-7f701a3437f84a84b6e3290266d24c94
I20240502 01:59:06.881862 29369 tablet_copy_source_session.cc:205] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.882335 4999 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.882550 4999 tablet_bootstrap.cc:492] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:59:06.882685 4999 ts_tablet_manager.cc:1367] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20240502 01:59:06.883070 4999 raft_consensus.cc:348] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:06.883291 4937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f701a3437f84a84b6e3290266d24c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.883289 4999 raft_consensus.cc:729] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.883437 4999 consensus_queue.cc:260] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:06.883751 4999 ts_tablet_manager.cc:1398] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:06.884471 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-b546a99821624f0588a4c0f59a5ec5a3 received from {username='slave'} at 127.19.43.200:53349
I20240502 01:59:06.884541 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-b546a99821624f0588a4c0f59a5ec5a3 on tablet b546a99821624f0588a4c0f59a5ec5a3 with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:06.885232 4937 tablet_copy_client.cc:783] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.885376 4937 tablet_copy_client.cc:647] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.886546 4937 tablet_copy_client.cc:523] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.887615 4937 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:06.891604 4937 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.891813 4937 tablet_bootstrap.cc:492] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:06.892104 4937 ts_tablet_manager.cc:1367] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 01:59:06.892479 4937 raft_consensus.cc:348] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.892587 4937 raft_consensus.cc:729] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.892742 4937 consensus_queue.cc:260] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:06.893020 4937 ts_tablet_manager.cc:1398] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:06.893501 29369 tablet_copy_service.cc:342] P 468dd27456c64d67b22e8a58af93d93d: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-7f701a3437f84a84b6e3290266d24c94 received from {username='slave'} at 127.19.43.195:43081
I20240502 01:59:06.893563 29369 tablet_copy_service.cc:433] P 468dd27456c64d67b22e8a58af93d93d: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-7f701a3437f84a84b6e3290266d24c94 on tablet 7f701a3437f84a84b6e3290266d24c94 with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:06.899010 5074 ts_tablet_manager.cc:902] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:06.899648 5074 tablet_copy_client.cc:308] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.200:36259
I20240502 01:59:06.900059 29512 tablet_copy_service.cc:140] P 7c693d0933c64d9babc07f404df45cd1: Received BeginTabletCopySession request for tablet 6a5f8bfeddd1443da789f67949830f99 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:46561)
I20240502 01:59:06.900153 29512 tablet_copy_service.cc:161] P 7c693d0933c64d9babc07f404df45cd1: Beginning new tablet copy session on tablet 6a5f8bfeddd1443da789f67949830f99 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:46561: session id = 468dd27456c64d67b22e8a58af93d93d-6a5f8bfeddd1443da789f67949830f99
I20240502 01:59:06.901042 29512 tablet_copy_source_session.cc:205] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.901751 5074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5f8bfeddd1443da789f67949830f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.904281 5074 tablet_copy_client.cc:783] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.904476 5074 tablet_copy_client.cc:647] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.905503 5074 tablet_copy_client.cc:523] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.906731 5074 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:06.910881 5074 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:06.911144 5074 tablet_bootstrap.cc:492] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:59:06.911311 5074 ts_tablet_manager.cc:1367] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 01:59:06.911721 5074 raft_consensus.cc:348] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:06.911892 5074 raft_consensus.cc:729] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:06.912068 5074 consensus_queue.cc:260] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:06.913137 5074 ts_tablet_manager.cc:1398] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:06.913518 29512 tablet_copy_service.cc:342] P 7c693d0933c64d9babc07f404df45cd1: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-6a5f8bfeddd1443da789f67949830f99 received from {username='slave'} at 127.19.43.199:46561
I20240502 01:59:06.913583 29512 tablet_copy_service.cc:433] P 7c693d0933c64d9babc07f404df45cd1: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-6a5f8bfeddd1443da789f67949830f99 on tablet 6a5f8bfeddd1443da789f67949830f99 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:59:06.936821 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:06.936939 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:06.939663 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 7317c932fdba4235816aff6d5c482878
I20240502 01:59:06.940197 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 7317c932fdba4235816aff6d5c482878 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:06 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:06.940186 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 7317c932fdba4235816aff6d5c482878 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:06 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:06.940483 5100 tablet_replica.cc:330] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:06.940629 5100 raft_consensus.cc:2229] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:06.940872 5100 raft_consensus.cc:2258] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:06.940861 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 7317c932fdba4235816aff6d5c482878 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:06 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:06.941895 26793 raft_consensus.cc:1203] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:06.941988 5100 ts_tablet_manager.cc:1875] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:06.947403 5100 ts_tablet_manager.cc:1888] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:06.947469 5100 log.cc:1196] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/7317c932fdba4235816aff6d5c482878
I20240502 01:59:06.947711 5100 ts_tablet_manager.cc:1909] T 7317c932fdba4235816aff6d5c482878 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:06.948313 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table_new_name [id=329f83c2a6144f59993e545d2b3209e3]) successfully deleted
I20240502 01:59:06.950160 5102 tablet_replica.cc:330] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:06.950366 5102 raft_consensus.cc:2229] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:06.950500 5102 raft_consensus.cc:2258] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:06.950716 5101 tablet_replica.cc:330] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:06.950838 5101 raft_consensus.cc:2229] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:06.950953 5101 raft_consensus.cc:2258] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:06.951400 5102 ts_tablet_manager.cc:1875] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:06.951437 5101 ts_tablet_manager.cc:1875] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:06.953650 5102 ts_tablet_manager.cc:1888] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:06.953754 5101 ts_tablet_manager.cc:1888] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:06.953769 5102 log.cc:1196] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/7317c932fdba4235816aff6d5c482878
I20240502 01:59:06.953897 5101 log.cc:1196] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/7317c932fdba4235816aff6d5c482878
I20240502 01:59:06.954149 5102 ts_tablet_manager.cc:1909] T 7317c932fdba4235816aff6d5c482878 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:06.954279 5101 ts_tablet_manager.cc:1909] T 7317c932fdba4235816aff6d5c482878 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:06.954845 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table_new_name [id=329f83c2a6144f59993e545d2b3209e3]) successfully deleted
I20240502 01:59:06.955089 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 7317c932fdba4235816aff6d5c482878 (table rebalancer_dynamic_table_new_name [id=329f83c2a6144f59993e545d2b3209e3]) successfully deleted
I20240502 01:59:06.971503 4999 ts_tablet_manager.cc:902] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:06.971907 4999 tablet_copy_client.cc:308] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:59:06.972709 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 626774246b9445bc838427358e08f2cd from peer 7c693d0933c64d9babc07f404df45cd1 ({username='slave'} at 127.19.43.200:45861)
I20240502 01:59:06.972810 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 626774246b9445bc838427358e08f2cd from peer 7c693d0933c64d9babc07f404df45cd1 at {username='slave'} at 127.19.43.200:45861: session id = 7c693d0933c64d9babc07f404df45cd1-626774246b9445bc838427358e08f2cd
I20240502 01:59:06.973829 26950 tablet_copy_source_session.cc:205] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:06.977056 4999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626774246b9445bc838427358e08f2cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:06.979038 4999 tablet_copy_client.cc:783] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:06.979177 4999 tablet_copy_client.cc:647] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:06.984395 4999 tablet_copy_client.cc:523] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:06.985666 4999 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:06.993332 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:06.994740 29215 raft_consensus.cc:1203] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 01:59:06.997541 5005 ts_tablet_manager.cc:902] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Initiating tablet copy from peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:06.997959 5005 tablet_copy_client.cc:308] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.196:43635
I20240502 01:59:06.998391 27083 tablet_copy_service.cc:140] P 68e5b1eeb04448fea818a14991eab87c: Received BeginTabletCopySession request for tablet e1950ace91824415b9bf3d1657bad72f from peer 0d83fdd1e6e04cf5a5602994d1dbc0df ({username='slave'} at 127.19.43.198:58871)
I20240502 01:59:06.998481 27083 tablet_copy_service.cc:161] P 68e5b1eeb04448fea818a14991eab87c: Beginning new tablet copy session on tablet e1950ace91824415b9bf3d1657bad72f from peer 0d83fdd1e6e04cf5a5602994d1dbc0df at {username='slave'} at 127.19.43.198:58871: session id = 0d83fdd1e6e04cf5a5602994d1dbc0df-e1950ace91824415b9bf3d1657bad72f
I20240502 01:59:06.999348 27083 tablet_copy_source_session.cc:205] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:07.000612 5005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1950ace91824415b9bf3d1657bad72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.002638 5005 tablet_copy_client.cc:783] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:07.005141 5005 tablet_copy_client.cc:647] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:07.007813 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:07.021136 26640 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:07.022385 29059 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:07.024895 26774 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:07.027112 4999 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:07.027403 4999 tablet_bootstrap.cc:492] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: Bootstrap complete.
I20240502 01:59:07.027547 4999 ts_tablet_manager.cc:1367] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.042s user 0.024s sys 0.001s
I20240502 01:59:07.027941 4999 raft_consensus.cc:348] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.028048 4999 raft_consensus.cc:729] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.028194 4999 consensus_queue.cc:260] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.029220 4999 ts_tablet_manager.cc:1398] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:07.029610 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 7c693d0933c64d9babc07f404df45cd1-626774246b9445bc838427358e08f2cd received from {username='slave'} at 127.19.43.200:45861
I20240502 01:59:07.029680 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 7c693d0933c64d9babc07f404df45cd1-626774246b9445bc838427358e08f2cd on tablet 626774246b9445bc838427358e08f2cd with peer 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:07.032205 26794 raft_consensus.cc:1203] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.033074 3993 consensus_queue.cc:1035] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.033948 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:07.044229 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:07.045672 3993 consensus_queue.cc:1035] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.049860 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb55c82075f48ba8d50c004804c29da (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e]), partition=
I20240502 01:59:07.050114 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb55c82075f48ba8d50c004804c29da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.053560 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb55c82075f48ba8d50c004804c29da (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e]), partition=
I20240502 01:59:07.053633 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb55c82075f48ba8d50c004804c29da (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e]), partition=
I20240502 01:59:07.053778 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb55c82075f48ba8d50c004804c29da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.053815 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb55c82075f48ba8d50c004804c29da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.054951 5126 tablet_bootstrap.cc:492] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:07.055794 5126 tablet_bootstrap.cc:654] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.055927 5127 tablet_bootstrap.cc:492] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:07.056284 29329 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:07.057196 5127 tablet_bootstrap.cc:654] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.058233 5127 tablet_bootstrap.cc:492] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:07.058316 5127 ts_tablet_manager.cc:1367] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:07.058743 5127 raft_consensus.cc:348] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.058856 5127 raft_consensus.cc:374] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.058887 5127 raft_consensus.cc:729] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.059000 5127 consensus_queue.cc:260] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.059295 5127 ts_tablet_manager.cc:1398] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.064976 5126 tablet_bootstrap.cc:492] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:07.065145 5126 ts_tablet_manager.cc:1367] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 01:59:07.073835 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:07.078192 5005 tablet_copy_client.cc:523] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:07.079609 5005 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:07.082921 5126 raft_consensus.cc:348] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.082989 29487 raft_consensus.cc:1203] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.083492 5128 tablet_bootstrap.cc:492] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:07.085382 5126 raft_consensus.cc:374] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.085461 5126 raft_consensus.cc:729] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.085608 5126 consensus_queue.cc:260] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.086381 5126 ts_tablet_manager.cc:1398] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.021s user 0.001s sys 0.000s
I20240502 01:59:07.087030 5128 tablet_bootstrap.cc:654] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.091552 5133 ts_tablet_manager.cc:902] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: Initiating tablet copy from peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:07.092031 5133 tablet_copy_client.cc:308] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.200:36259
I20240502 01:59:07.093550 5128 tablet_bootstrap.cc:492] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:07.093694 5128 ts_tablet_manager.cc:1367] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 01:59:07.094332 5128 raft_consensus.cc:348] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.094491 5128 raft_consensus.cc:374] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.094558 5128 raft_consensus.cc:729] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.094720 5128 consensus_queue.cc:260] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.095469 29213 raft_consensus.cc:1203] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 01:59:07.096457 5128 ts_tablet_manager.cc:1398] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:07.097019 29512 tablet_copy_service.cc:140] P 7c693d0933c64d9babc07f404df45cd1: Received BeginTabletCopySession request for tablet 9ca8563ff68c4fa590c17ef58e67afa0 from peer 68e5b1eeb04448fea818a14991eab87c ({username='slave'} at 127.19.43.196:55395)
I20240502 01:59:07.097116 29512 tablet_copy_service.cc:161] P 7c693d0933c64d9babc07f404df45cd1: Beginning new tablet copy session on tablet 9ca8563ff68c4fa590c17ef58e67afa0 from peer 68e5b1eeb04448fea818a14991eab87c at {username='slave'} at 127.19.43.196:55395: session id = 68e5b1eeb04448fea818a14991eab87c-9ca8563ff68c4fa590c17ef58e67afa0
I20240502 01:59:07.097956 29512 tablet_copy_source_session.cc:205] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:07.098915 5133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca8563ff68c4fa590c17ef58e67afa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.101329 5133 tablet_copy_client.cc:783] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:07.101482 5133 tablet_copy_client.cc:647] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:07.107504 5133 tablet_copy_client.cc:523] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:07.107583 27064 raft_consensus.cc:1203] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.108646 5133 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:07.115645 29213 raft_consensus.cc:1203] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 01:59:07.115700 5133 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:07.115916 5133 tablet_bootstrap.cc:492] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap complete.
I20240502 01:59:07.116076 5133 ts_tablet_manager.cc:1367] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20240502 01:59:07.116495 5133 raft_consensus.cc:348] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.116607 5133 raft_consensus.cc:729] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.116724 5133 consensus_queue.cc:260] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.117350 5133 ts_tablet_manager.cc:1398] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 01:59:07.117748 26930 raft_consensus.cc:1203] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.118847 29512 tablet_copy_service.cc:342] P 7c693d0933c64d9babc07f404df45cd1: Request end of tablet copy session 68e5b1eeb04448fea818a14991eab87c-9ca8563ff68c4fa590c17ef58e67afa0 received from {username='slave'} at 127.19.43.196:55395
I20240502 01:59:07.118911 29512 tablet_copy_service.cc:433] P 7c693d0933c64d9babc07f404df45cd1: ending tablet copy session 68e5b1eeb04448fea818a14991eab87c-9ca8563ff68c4fa590c17ef58e67afa0 on tablet 9ca8563ff68c4fa590c17ef58e67afa0 with peer 68e5b1eeb04448fea818a14991eab87c
I20240502 01:59:07.119580 5005 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:07.119827 5005 tablet_bootstrap.cc:492] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap complete.
I20240502 01:59:07.119979 5005 ts_tablet_manager.cc:1367] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.040s user 0.022s sys 0.000s
I20240502 01:59:07.120518 5005 raft_consensus.cc:348] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:07.120632 5005 raft_consensus.cc:729] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.120918 5005 consensus_queue.cc:260] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:07.121939 5005 ts_tablet_manager.cc:1398] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:07.122339 27083 tablet_copy_service.cc:342] P 68e5b1eeb04448fea818a14991eab87c: Request end of tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-e1950ace91824415b9bf3d1657bad72f received from {username='slave'} at 127.19.43.198:58871
I20240502 01:59:07.122409 27083 tablet_copy_service.cc:433] P 68e5b1eeb04448fea818a14991eab87c: ending tablet copy session 0d83fdd1e6e04cf5a5602994d1dbc0df-e1950ace91824415b9bf3d1657bad72f on tablet e1950ace91824415b9bf3d1657bad72f with peer 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:59:07.129833 26659 raft_consensus.cc:1203] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.142570 4551 raft_consensus.cc:482] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:07.142663 4551 raft_consensus.cc:504] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.143000 4551 leader_election.cc:288] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:07.143257 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb55c82075f48ba8d50c004804c29da" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:07.143371 29079 raft_consensus.cc:2454] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:07.143365 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb55c82075f48ba8d50c004804c29da" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:07.143474 26930 raft_consensus.cc:2454] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:07.143635 29283 leader_election.cc:302] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:07.144138 4530 raft_consensus.cc:2781] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:07.144191 4530 raft_consensus.cc:482] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:07.144223 4530 raft_consensus.cc:3037] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.145140 4530 raft_consensus.cc:504] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.145453 4530 leader_election.cc:288] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:07.145865 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb55c82075f48ba8d50c004804c29da" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:07.145958 26930 raft_consensus.cc:3037] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.146859 26930 raft_consensus.cc:2454] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:07.147284 29286 leader_election.cc:302] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:07.147615 4551 raft_consensus.cc:2781] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:07.147698 4551 raft_consensus.cc:686] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:07.147814 4551 consensus_queue.cc:237] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:07.148012 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb55c82075f48ba8d50c004804c29da" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:07.148105 29079 raft_consensus.cc:3037] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.149061 29079 raft_consensus.cc:2454] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:07.150224 26515 catalog_manager.cc:5484] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:07.153604 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:07.160181 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 027b2e9a4fe74a2fb6f8e204e7f20c28 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.160461 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 027b2e9a4fe74a2fb6f8e204e7f20c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.160758 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 7b3008a463fc411caa73cfef95e614f2 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.160977 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b3008a463fc411caa73cfef95e614f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.160993 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 027b2e9a4fe74a2fb6f8e204e7f20c28 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.161254 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 027b2e9a4fe74a2fb6f8e204e7f20c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.162909 5127 tablet_bootstrap.cc:492] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:07.163146 5126 tablet_bootstrap.cc:492] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:07.163434 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 027b2e9a4fe74a2fb6f8e204e7f20c28 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.163684 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 027b2e9a4fe74a2fb6f8e204e7f20c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.163729 5127 tablet_bootstrap.cc:654] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.163996 5126 tablet_bootstrap.cc:654] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.164666 27044 tablet_service.cc:1467] Processing CreateTablet for tablet b059fe85a59e41ac8ef27c669476521e (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.164686 26641 tablet_service.cc:1467] Processing CreateTablet for tablet b059fe85a59e41ac8ef27c669476521e (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.164934 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b059fe85a59e41ac8ef27c669476521e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.164940 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b059fe85a59e41ac8ef27c669476521e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.166499 5127 tablet_bootstrap.cc:492] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:07.166580 5127 ts_tablet_manager.cc:1367] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:07.167035 5127 raft_consensus.cc:348] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.167168 5127 raft_consensus.cc:374] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.167200 5127 raft_consensus.cc:729] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.167312 5127 consensus_queue.cc:260] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.167615 5127 ts_tablet_manager.cc:1398] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.168996 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 7b3008a463fc411caa73cfef95e614f2 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.169225 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b3008a463fc411caa73cfef95e614f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.169860 29079 raft_consensus.cc:1203] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 01:59:07.171403 5127 tablet_bootstrap.cc:492] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:07.172231 5127 tablet_bootstrap.cc:654] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.173018 29059 tablet_service.cc:1467] Processing CreateTablet for tablet b059fe85a59e41ac8ef27c669476521e (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.173379 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 7b3008a463fc411caa73cfef95e614f2 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=43ca2c0594d64e3dad0ec4fbbc4899b0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.173606 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b3008a463fc411caa73cfef95e614f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.173899 5127 tablet_bootstrap.cc:492] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:07.173975 5127 ts_tablet_manager.cc:1367] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:07.174413 5127 raft_consensus.cc:348] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.174503 5127 raft_consensus.cc:374] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.174533 5127 raft_consensus.cc:729] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.174638 5127 consensus_queue.cc:260] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.174896 5127 ts_tablet_manager.cc:1398] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:07.175091 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b059fe85a59e41ac8ef27c669476521e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.177685 5161 tablet_bootstrap.cc:492] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:07.178045 5127 tablet_bootstrap.cc:492] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:07.178149 5160 tablet_bootstrap.cc:492] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:07.178932 5127 tablet_bootstrap.cc:654] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.179131 5161 tablet_bootstrap.cc:654] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.180009 5160 tablet_bootstrap.cc:654] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.180037 5126 tablet_bootstrap.cc:492] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:07.180114 5126 ts_tablet_manager.cc:1367] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:59:07.180560 5126 raft_consensus.cc:348] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.180667 5126 raft_consensus.cc:374] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.180698 5126 raft_consensus.cc:729] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.180842 5126 consensus_queue.cc:260] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.181123 5126 ts_tablet_manager.cc:1398] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.181277 5126 tablet_bootstrap.cc:492] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:07.181696 5160 tablet_bootstrap.cc:492] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:07.181777 5160 ts_tablet_manager.cc:1367] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:07.181859 4592 raft_consensus.cc:482] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:07.181924 4592 raft_consensus.cc:504] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.182224 5160 raft_consensus.cc:348] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.182297 4592 leader_election.cc:288] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:07.182332 5160 raft_consensus.cc:374] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.182361 5160 raft_consensus.cc:729] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.182464 5160 consensus_queue.cc:260] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.182734 5160 ts_tablet_manager.cc:1398] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.182793 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b3008a463fc411caa73cfef95e614f2" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:07.182876 5160 tablet_bootstrap.cc:492] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:07.183169 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b3008a463fc411caa73cfef95e614f2" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:07.183271 26930 raft_consensus.cc:2454] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:07.183730 5160 tablet_bootstrap.cc:654] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.183768 5126 tablet_bootstrap.cc:654] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.183907 29016 leader_election.cc:302] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:07.184077 4592 raft_consensus.cc:2781] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:07.184136 4592 raft_consensus.cc:482] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:07.184172 4592 raft_consensus.cc:3037] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.184669 5161 tablet_bootstrap.cc:492] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:07.184815 5161 ts_tablet_manager.cc:1367] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
W20240502 01:59:07.184088 29016 leader_election.cc:341] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:07.185101 4592 raft_consensus.cc:504] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.185434 4592 leader_election.cc:288] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:07.185937 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b3008a463fc411caa73cfef95e614f2" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:07.186326 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b3008a463fc411caa73cfef95e614f2" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:07.186331 5161 raft_consensus.cc:348] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.186412 5161 raft_consensus.cc:374] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.186422 26930 raft_consensus.cc:3037] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.186444 5161 raft_consensus.cc:729] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.186535 5161 consensus_queue.cc:260] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.186980 5161 ts_tablet_manager.cc:1398] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:07.187170 5161 tablet_bootstrap.cc:492] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:07.187342 26930 raft_consensus.cc:2454] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:07.188004 5161 tablet_bootstrap.cc:654] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.188720 29016 leader_election.cc:302] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
W20240502 01:59:07.188910 29016 leader_election.cc:341] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:07.189235 5126 tablet_bootstrap.cc:492] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:07.189313 5126 ts_tablet_manager.cc:1367] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:59:07.189594 4592 raft_consensus.cc:2781] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:07.189671 4592 raft_consensus.cc:686] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:07.189724 5160 tablet_bootstrap.cc:492] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:07.189678 5126 raft_consensus.cc:348] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.189776 5126 raft_consensus.cc:374] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.189792 5160 ts_tablet_manager.cc:1367] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:59:07.189808 5126 raft_consensus.cc:729] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.189791 4592 consensus_queue.cc:237] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.189932 5126 consensus_queue.cc:260] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.190215 5126 ts_tablet_manager.cc:1398] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.190228 5160 raft_consensus.cc:348] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.190335 5160 raft_consensus.cc:374] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.190364 5160 raft_consensus.cc:729] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.190469 5160 consensus_queue.cc:260] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.190742 5160 ts_tablet_manager.cc:1398] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.191479 5161 tablet_bootstrap.cc:492] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:07.191604 5161 ts_tablet_manager.cc:1367] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:07.192481 26515 catalog_manager.cc:5484] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:07.194181 5127 tablet_bootstrap.cc:492] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:07.194146 5161 raft_consensus.cc:348] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.194278 5127 ts_tablet_manager.cc:1367] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 01:59:07.194345 5161 raft_consensus.cc:374] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.194429 5161 raft_consensus.cc:729] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.194581 5161 consensus_queue.cc:260] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.194674 5127 raft_consensus.cc:348] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.194775 5127 raft_consensus.cc:374] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.194815 5127 raft_consensus.cc:729] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.194890 26793 raft_consensus.cc:1203] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.194914 5127 consensus_queue.cc:260] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.195190 5161 ts_tablet_manager.cc:1398] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:07.195195 5127 ts_tablet_manager.cc:1398] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.220676 3791 raft_consensus.cc:482] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:07.220785 3791 raft_consensus.cc:504] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:07.221136 3791 leader_election.cc:288] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:07.221810 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b3008a463fc411caa73cfef95e614f2" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:07.221961 26930 raft_consensus.cc:2379] T 7b3008a463fc411caa73cfef95e614f2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68e5b1eeb04448fea818a14991eab87c in current term 1: Already voted for candidate f3d18a019013487ab801599cfb80e4b4 in this term.
I20240502 01:59:07.222465 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b3008a463fc411caa73cfef95e614f2" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:07.222819 26995 leader_election.cc:302] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:07.223037 3791 raft_consensus.cc:3037] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.224282 3791 raft_consensus.cc:2726] T 7b3008a463fc411caa73cfef95e614f2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 01:59:07.274230 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:07.277930 26930 raft_consensus.cc:1261] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:07.277979 29079 raft_consensus.cc:1261] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:07.278358 4530 consensus_queue.cc:1035] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.278525 5151 consensus_queue.cc:1035] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.281255 26515 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 7eb55c82075f48ba8d50c004804c29da (table rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e]). Expected version 1 got 0
I20240502 01:59:07.282783 26514 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet 7eb55c82075f48ba8d50c004804c29da (table rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e]). Expected version 1 got 0
I20240502 01:59:07.282469 5198 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:07.283267 5198 tablet.cc:1620] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:07.284271 5200 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:07.284551 5200 tablet.cc:1620] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:07.285594 26514 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet 7eb55c82075f48ba8d50c004804c29da (table rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e]). Expected version 1 got 0
I20240502 01:59:07.285728 5199 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:07.286010 5199 tablet.cc:1620] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:07.288062 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e] alter complete (version 1)
I20240502 01:59:07.303617 29349 raft_consensus.cc:1203] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.362149 29487 raft_consensus.cc:1203] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 01:59:07.372175 26930 raft_consensus.cc:1203] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.372694 4592 raft_consensus.cc:482] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:07.372787 4592 raft_consensus.cc:504] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.373162 4592 leader_election.cc:288] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:07.373437 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027b2e9a4fe74a2fb6f8e204e7f20c28" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:07.373574 26659 raft_consensus.cc:2454] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:07.373746 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027b2e9a4fe74a2fb6f8e204e7f20c28" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:07.373865 26930 raft_consensus.cc:2454] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:07.373844 29015 leader_election.cc:302] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:07.374044 4592 raft_consensus.cc:2781] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:07.374099 4592 raft_consensus.cc:482] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:07.374137 4592 raft_consensus.cc:3037] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.375175 4592 raft_consensus.cc:504] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.375499 4592 leader_election.cc:288] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:07.375864 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027b2e9a4fe74a2fb6f8e204e7f20c28" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:07.375989 26659 raft_consensus.cc:3037] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.376888 26659 raft_consensus.cc:2454] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:07.377357 29015 leader_election.cc:302] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:07.377518 4592 raft_consensus.cc:2781] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:07.377547 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027b2e9a4fe74a2fb6f8e204e7f20c28" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:07.377610 4592 raft_consensus.cc:686] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:07.377633 26930 raft_consensus.cc:3037] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.377751 4592 consensus_queue.cc:237] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:07.378544 26930 raft_consensus.cc:2454] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:07.380825 26514 catalog_manager.cc:5484] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:07.391273 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:07.395850 5198 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:07.396138 5198 tablet.cc:1620] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:07.396445 5200 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:07.397332 5200 tablet.cc:1620] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:07.397626 5199 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:07.397954 5199 tablet.cc:1620] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:07.401590 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=c3db9ae57df54762aed71b2f8dade80e] alter complete (version 2)
I20240502 01:59:07.415613 3791 raft_consensus.cc:482] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:07.415700 3791 raft_consensus.cc:504] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.416024 3791 leader_election.cc:288] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:07.416291 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b059fe85a59e41ac8ef27c669476521e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:07.416409 29079 raft_consensus.cc:2454] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:07.416396 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b059fe85a59e41ac8ef27c669476521e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:07.416519 26659 raft_consensus.cc:2454] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:07.416630 26995 leader_election.cc:302] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:07.416790 3791 raft_consensus.cc:2781] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:07.416844 3791 raft_consensus.cc:482] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:07.416880 3791 raft_consensus.cc:3037] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.417620 3791 raft_consensus.cc:504] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.417915 3791 leader_election.cc:288] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:07.418254 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b059fe85a59e41ac8ef27c669476521e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:07.418262 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b059fe85a59e41ac8ef27c669476521e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:07.418359 26659 raft_consensus.cc:3037] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.418359 29079 raft_consensus.cc:3037] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:07.419252 26659 raft_consensus.cc:2454] T b059fe85a59e41ac8ef27c669476521e P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:07.419263 29079 raft_consensus.cc:2454] T b059fe85a59e41ac8ef27c669476521e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:07.419512 26997 leader_election.cc:302] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:07.419715 3791 raft_consensus.cc:2781] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:07.419812 3791 raft_consensus.cc:686] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:07.419957 3791 consensus_queue.cc:237] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:07.421658 26515 catalog_manager.cc:5484] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:07.451360 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:07.451762 29329 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:07.451979 27044 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:07.452315 29487 raft_consensus.cc:1203] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.452450 29213 raft_consensus.cc:1203] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.452534 26930 raft_consensus.cc:1203] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.457924 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:07.463213 26774 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:07.467556 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:07.482523 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:07.482786 26641 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:07.504644 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:07.510802 5198 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:07.511582 5199 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:07.513188 5200 tablet.cc:1722] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:07.523620 29215 raft_consensus.cc:1203] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 01:59:07.522658 26514 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=c3db9ae57df54762aed71b2f8dade80e] alter complete (version 3)
I20240502 01:59:07.539860 26659 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564"
dest_uuid: "998b57e2badc4d779c586c6579149114"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46220
I20240502 01:59:07.540020 26659 raft_consensus.cc:595] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Received request to transfer leadership
I20240502 01:59:07.624208 26508 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:07.624346 26508 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:07.627460 26508 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 7eb55c82075f48ba8d50c004804c29da
I20240502 01:59:07.628059 29329 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb55c82075f48ba8d50c004804c29da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:07 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:07.628333 5257 tablet_replica.cc:330] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:07.628512 5257 raft_consensus.cc:2229] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:07.628727 5257 raft_consensus.cc:2258] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:07.628755 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb55c82075f48ba8d50c004804c29da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:07 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:07.629321 5258 tablet_replica.cc:330] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:07.629431 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb55c82075f48ba8d50c004804c29da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:07 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:07.629596 5258 raft_consensus.cc:2229] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:07.629737 5259 tablet_replica.cc:330] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:07.629755 5257 ts_tablet_manager.cc:1875] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:07.629807 5258 raft_consensus.cc:2258] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:07.630475 5258 ts_tablet_manager.cc:1875] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:07.632294 5257 ts_tablet_manager.cc:1888] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:07.632398 5257 log.cc:1196] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/7eb55c82075f48ba8d50c004804c29da
I20240502 01:59:07.632465 5259 raft_consensus.cc:2229] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:07.632606 5259 raft_consensus.cc:2258] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:07.632658 5257 ts_tablet_manager.cc:1909] T 7eb55c82075f48ba8d50c004804c29da P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:59:07.634536 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 7eb55c82075f48ba8d50c004804c29da (table rebalancer_dynamic_table_new_name [id=c3db9ae57df54762aed71b2f8dade80e]) successfully deleted
I20240502 01:59:07.634585 5259 ts_tablet_manager.cc:1875] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:07.637215 5258 ts_tablet_manager.cc:1888] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:07.637336 5258 log.cc:1196] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/7eb55c82075f48ba8d50c004804c29da
I20240502 01:59:07.643126 5259 ts_tablet_manager.cc:1888] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:07.643199 5259 log.cc:1196] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/7eb55c82075f48ba8d50c004804c29da
I20240502 01:59:07.643486 5259 ts_tablet_manager.cc:1909] T 7eb55c82075f48ba8d50c004804c29da P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:59:07.643534 5258 ts_tablet_manager.cc:1909] T 7eb55c82075f48ba8d50c004804c29da P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:07.644165 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 7eb55c82075f48ba8d50c004804c29da (table rebalancer_dynamic_table_new_name [id=c3db9ae57df54762aed71b2f8dade80e]) successfully deleted
I20240502 01:59:07.644732 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 7eb55c82075f48ba8d50c004804c29da (table rebalancer_dynamic_table_new_name [id=c3db9ae57df54762aed71b2f8dade80e]) successfully deleted
I20240502 01:59:07.662379 27064 raft_consensus.cc:1203] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:07.673656 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:07.675689 29329 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:07.681551 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:07.686614 26774 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:07.692096 26641 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:07.693104 4592 consensus_queue.cc:1035] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.702286 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:07.704262 4592 consensus_queue.cc:1035] T 7b3008a463fc411caa73cfef95e614f2 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.711717 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:07.713311 27044 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:07.730520 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:07.737128 26641 tablet_service.cc:1467] Processing CreateTablet for tablet a9b2529e62b3426698b9e17a1ea472fc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]), partition=
I20240502 01:59:07.737375 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9b2529e62b3426698b9e17a1ea472fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.739823 26774 tablet_service.cc:1467] Processing CreateTablet for tablet a9b2529e62b3426698b9e17a1ea472fc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]), partition=
I20240502 01:59:07.740056 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9b2529e62b3426698b9e17a1ea472fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.743027 29454 tablet_service.cc:1467] Processing CreateTablet for tablet a9b2529e62b3426698b9e17a1ea472fc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]), partition=
I20240502 01:59:07.743261 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9b2529e62b3426698b9e17a1ea472fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.745774 5274 tablet_bootstrap.cc:492] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:07.746140 5275 tablet_bootstrap.cc:492] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:07.748370 5275 tablet_bootstrap.cc:654] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.748620 5276 tablet_bootstrap.cc:492] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:07.749713 5276 tablet_bootstrap.cc:654] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.750286 5274 tablet_bootstrap.cc:654] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.751545 5276 tablet_bootstrap.cc:492] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:07.751637 5276 ts_tablet_manager.cc:1367] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:07.751631 5274 tablet_bootstrap.cc:492] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:07.751706 5274 ts_tablet_manager.cc:1367] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:07.752027 5276 raft_consensus.cc:348] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.752126 5276 raft_consensus.cc:374] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.752153 5276 raft_consensus.cc:729] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.752267 5276 consensus_queue.cc:260] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.752130 5274 raft_consensus.cc:348] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.752362 5274 raft_consensus.cc:374] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.752406 5274 raft_consensus.cc:729] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.752547 5276 ts_tablet_manager.cc:1398] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:07.752725 5274 consensus_queue.cc:260] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.753217 5274 ts_tablet_manager.cc:1398] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.753687 5275 tablet_bootstrap.cc:492] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:07.753773 5275 ts_tablet_manager.cc:1367] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:59:07.754204 5275 raft_consensus.cc:348] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.754293 5275 raft_consensus.cc:374] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.754317 5275 raft_consensus.cc:729] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.754417 5275 consensus_queue.cc:260] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:07.754691 5275 ts_tablet_manager.cc:1398] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.872071 4649 raft_consensus.cc:979] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: : Instructing follower b290c1bf893444078ed721f25995d124 to start an election
I20240502 01:59:07.872193 4649 raft_consensus.cc:1067] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Signalling peer b290c1bf893444078ed721f25995d124 to start an election
I20240502 01:59:07.872699 26793 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564"
dest_uuid: "b290c1bf893444078ed721f25995d124"
from {username='slave'} at 127.19.43.193:37351
I20240502 01:59:07.872880 26793 raft_consensus.cc:482] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:07.872951 26793 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:07.873809 26793 raft_consensus.cc:504] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:07.874305 26793 leader_election.cc:288] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:07.874505 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:07.874614 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:07.874644 29080 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:07.874723 26659 raft_consensus.cc:3032] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:07.874775 26659 raft_consensus.cc:729] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:07.874745 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d216b8ccd8ea43df85c1013cdaf36564" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:07.874873 29213 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:07.874936 26659 consensus_queue.cc:260] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:07.875128 26659 raft_consensus.cc:3037] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:07.875610 29080 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 2.
I20240502 01:59:07.875878 29213 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 2.
I20240502 01:59:07.876056 26659 raft_consensus.cc:2454] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 2.
I20240502 01:59:07.876117 26729 leader_election.cc:302] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:07.876257 4734 raft_consensus.cc:2781] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:07.876394 4734 raft_consensus.cc:686] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 2 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:07.876526 4734 consensus_queue.cc:237] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:07.878903 26514 catalog_manager.cc:5484] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 1 to 2, leader changed from 998b57e2badc4d779c586c6579149114 (127.19.43.193) to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:07.899087 2712 consensus_queue.cc:1035] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.901675 2712 consensus_queue.cc:1035] T 027b2e9a4fe74a2fb6f8e204e7f20c28 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.937781 4345 consensus_queue.cc:1035] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.941058 3791 consensus_queue.cc:1035] T b059fe85a59e41ac8ef27c669476521e P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:07.975694 26514 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:07.982231 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 7cd36fd57d0646389b4db76561aaf500 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.982517 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cd36fd57d0646389b4db76561aaf500. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.984571 5274 tablet_bootstrap.cc:492] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:07.985471 5274 tablet_bootstrap.cc:654] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.985723 29195 tablet_service.cc:1467] Processing CreateTablet for tablet b80c93e207b64e01bd62338e133da1d4 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.985975 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80c93e207b64e01bd62338e133da1d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.986865 27044 tablet_service.cc:1467] Processing CreateTablet for tablet b80c93e207b64e01bd62338e133da1d4 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.987125 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80c93e207b64e01bd62338e133da1d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.989310 5331 tablet_bootstrap.cc:492] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:07.990196 5331 tablet_bootstrap.cc:654] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.991696 29329 tablet_service.cc:1467] Processing CreateTablet for tablet b80c93e207b64e01bd62338e133da1d4 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.991946 5274 tablet_bootstrap.cc:492] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:07.991950 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80c93e207b64e01bd62338e133da1d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.992019 5274 ts_tablet_manager.cc:1367] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:07.992452 5274 raft_consensus.cc:348] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:07.992548 5274 raft_consensus.cc:374] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:07.992574 5274 raft_consensus.cc:729] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:07.992687 5274 consensus_queue.cc:260] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:07.992964 5274 ts_tablet_manager.cc:1398] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:07.994179 5334 tablet_bootstrap.cc:492] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:07.994367 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 7cd36fd57d0646389b4db76561aaf500 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.994572 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cd36fd57d0646389b4db76561aaf500. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.994640 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 57c7ef32ce164bcc9976a1a6177df0de (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.994639 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 7cd36fd57d0646389b4db76561aaf500 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.994879 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c7ef32ce164bcc9976a1a6177df0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.994879 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cd36fd57d0646389b4db76561aaf500. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.995414 5334 tablet_bootstrap.cc:654] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.996948 5275 tablet_bootstrap.cc:492] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:07.997475 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 57c7ef32ce164bcc9976a1a6177df0de (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.997684 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c7ef32ce164bcc9976a1a6177df0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:07.997737 5275 tablet_bootstrap.cc:654] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.998073 5336 tablet_bootstrap.cc:492] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:07.998998 5336 tablet_bootstrap.cc:654] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:07.999689 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 57c7ef32ce164bcc9976a1a6177df0de (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=958197952c18468b82c0ce094c2636cc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:07.999866 5330 tablet_bootstrap.cc:492] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:07.999929 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c7ef32ce164bcc9976a1a6177df0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.000128 5275 tablet_bootstrap.cc:492] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:08.000200 5275 ts_tablet_manager.cc:1367] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:08.000576 5275 raft_consensus.cc:348] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.000695 5275 raft_consensus.cc:374] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.000725 5275 raft_consensus.cc:729] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.000835 5275 consensus_queue.cc:260] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.000968 5330 tablet_bootstrap.cc:654] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.001097 5275 ts_tablet_manager.cc:1398] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.001762 5331 tablet_bootstrap.cc:492] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:08.001835 5331 ts_tablet_manager.cc:1367] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:59:08.001924 5336 tablet_bootstrap.cc:492] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:08.002053 5336 ts_tablet_manager.cc:1367] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:08.002249 5331 raft_consensus.cc:348] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.002373 5331 raft_consensus.cc:374] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.002410 5331 raft_consensus.cc:729] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.002512 5331 consensus_queue.cc:260] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.002763 5331 ts_tablet_manager.cc:1398] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:08.002502 5336 raft_consensus.cc:348] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.002866 5336 raft_consensus.cc:374] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.002903 5336 raft_consensus.cc:729] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.003003 5336 consensus_queue.cc:260] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.003262 5336 ts_tablet_manager.cc:1398] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:08.003969 5331 tablet_bootstrap.cc:492] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:08.004825 5331 tablet_bootstrap.cc:654] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.005600 5334 tablet_bootstrap.cc:492] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:08.005709 5334 ts_tablet_manager.cc:1367] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:59:08.006182 5334 raft_consensus.cc:348] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.006305 5334 raft_consensus.cc:374] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.006357 5334 raft_consensus.cc:729] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.006479 5334 consensus_queue.cc:260] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.006783 5334 ts_tablet_manager.cc:1398] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.006814 5331 tablet_bootstrap.cc:492] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:08.006927 5331 ts_tablet_manager.cc:1367] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:08.006942 5334 tablet_bootstrap.cc:492] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:08.007375 5331 raft_consensus.cc:348] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.007496 5331 raft_consensus.cc:374] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.007550 5331 raft_consensus.cc:729] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.007701 5331 consensus_queue.cc:260] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.007990 5331 ts_tablet_manager.cc:1398] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.008486 5334 tablet_bootstrap.cc:654] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.009510 5334 tablet_bootstrap.cc:492] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:08.009608 5334 ts_tablet_manager.cc:1367] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:08.009814 5330 tablet_bootstrap.cc:492] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:08.009898 5330 ts_tablet_manager.cc:1367] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 01:59:08.010049 5334 raft_consensus.cc:348] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.010214 5334 raft_consensus.cc:374] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.010294 5334 raft_consensus.cc:729] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.010382 5330 raft_consensus.cc:348] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.010496 5330 raft_consensus.cc:374] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.010447 5334 consensus_queue.cc:260] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.010563 5330 raft_consensus.cc:729] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.010697 5330 consensus_queue.cc:260] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.010758 5334 ts_tablet_manager.cc:1398] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:08.010936 5334 tablet_bootstrap.cc:492] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:08.011744 5334 tablet_bootstrap.cc:654] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.011075 5330 ts_tablet_manager.cc:1398] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.013590 5334 tablet_bootstrap.cc:492] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:08.013660 5334 ts_tablet_manager.cc:1367] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:59:08.014019 5334 raft_consensus.cc:348] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.014145 5334 raft_consensus.cc:374] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.014180 5334 raft_consensus.cc:729] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.014286 5334 consensus_queue.cc:260] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.014539 5334 ts_tablet_manager.cc:1398] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.053809 4649 raft_consensus.cc:482] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:08.053898 4649 raft_consensus.cc:504] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.054255 4649 leader_election.cc:288] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:08.054548 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd36fd57d0646389b4db76561aaf500" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:08.054651 29349 raft_consensus.cc:2454] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:08.054857 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd36fd57d0646389b4db76561aaf500" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:08.054913 26592 leader_election.cc:302] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:08.054963 26793 raft_consensus.cc:2454] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:08.055078 4649 raft_consensus.cc:2781] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:08.055145 4649 raft_consensus.cc:482] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:08.055193 4649 raft_consensus.cc:3037] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.055970 4649 raft_consensus.cc:504] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.056277 4649 leader_election.cc:288] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:08.056510 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd36fd57d0646389b4db76561aaf500" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:08.056546 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cd36fd57d0646389b4db76561aaf500" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:08.056620 26793 raft_consensus.cc:3037] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.056638 29349 raft_consensus.cc:3037] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.057391 26793 raft_consensus.cc:2454] T 7cd36fd57d0646389b4db76561aaf500 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:08.057456 29349 raft_consensus.cc:2454] T 7cd36fd57d0646389b4db76561aaf500 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:08.057643 26594 leader_election.cc:302] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:08.057786 4649 raft_consensus.cc:2781] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:08.057869 4649 raft_consensus.cc:686] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:08.057993 4649 consensus_queue.cc:237] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.059854 26511 catalog_manager.cc:5484] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:08.102135 4649 raft_consensus.cc:482] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:08.102538 4649 raft_consensus.cc:504] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:08.102891 4649 leader_election.cc:288] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:08.103128 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9b2529e62b3426698b9e17a1ea472fc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:08.103163 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9b2529e62b3426698b9e17a1ea472fc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:08.103233 26794 raft_consensus.cc:2454] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:08.103276 29487 raft_consensus.cc:2454] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:08.103518 26595 leader_election.cc:302] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:08.103720 4649 raft_consensus.cc:2781] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:08.103775 4649 raft_consensus.cc:482] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:08.103801 4649 raft_consensus.cc:3037] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.104527 4649 raft_consensus.cc:504] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:08.104821 4649 leader_election.cc:288] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:08.105070 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9b2529e62b3426698b9e17a1ea472fc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:08.105072 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9b2529e62b3426698b9e17a1ea472fc" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:08.105172 29487 raft_consensus.cc:3037] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.105181 26794 raft_consensus.cc:3037] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.106132 29487 raft_consensus.cc:2454] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:08.106130 26794 raft_consensus.cc:2454] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:08.106384 26595 leader_election.cc:302] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:08.106600 4649 raft_consensus.cc:2781] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:08.106707 4649 raft_consensus.cc:686] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:08.106824 4649 consensus_queue.cc:237] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:08.108942 26511 catalog_manager.cc:5484] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:08.123895 5344 raft_consensus.cc:482] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:08.123991 5344 raft_consensus.cc:504] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.124400 5344 leader_election.cc:288] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:08.124887 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c7ef32ce164bcc9976a1a6177df0de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:08.125031 29079 raft_consensus.cc:2454] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:08.125325 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c7ef32ce164bcc9976a1a6177df0de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:08.125459 27064 raft_consensus.cc:2454] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:08.125556 29283 leader_election.cc:302] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:08.125694 5344 raft_consensus.cc:2781] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:08.125749 5344 raft_consensus.cc:482] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:08.125800 5344 raft_consensus.cc:3037] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.126821 5344 raft_consensus.cc:504] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.127171 5344 leader_election.cc:288] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:08.127737 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c7ef32ce164bcc9976a1a6177df0de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:08.127844 29079 raft_consensus.cc:3037] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.128731 29079 raft_consensus.cc:2454] T 57c7ef32ce164bcc9976a1a6177df0de P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:08.128970 29283 leader_election.cc:302] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:08.129127 5344 raft_consensus.cc:2781] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:08.129221 5344 raft_consensus.cc:686] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:08.129344 5344 consensus_queue.cc:237] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:08.129784 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c7ef32ce164bcc9976a1a6177df0de" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:08.129892 27064 raft_consensus.cc:3037] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.130851 27064 raft_consensus.cc:2454] T 57c7ef32ce164bcc9976a1a6177df0de P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:08.134413 26511 catalog_manager.cc:5484] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:08.137570 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:08.144886 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.146878 27044 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:08.152465 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:08.153158 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:08.153187 26641 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:08.156497 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:08.180402 29329 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:08.235013 29489 consensus_queue.cc:237] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.236177 29350 raft_consensus.cc:1261] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:08.236526 26927 raft_consensus.cc:1261] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:08.236631 5375 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.236904 4005 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.238386 4005 raft_consensus.cc:2932] T 6a5f8bfeddd1443da789f67949830f99 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:08.238898 29350 raft_consensus.cc:2932] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:08.242368 26511 catalog_manager.cc:5484] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 2 to 3, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:08.242597 4551 raft_consensus.cc:482] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:08.242679 4551 raft_consensus.cc:504] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.243013 4551 leader_election.cc:288] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:08.243351 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80c93e207b64e01bd62338e133da1d4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:08.243466 27064 raft_consensus.cc:2454] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:08.243412 26927 raft_consensus.cc:2932] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:08.243765 29286 leader_election.cc:302] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:08.243921 4551 raft_consensus.cc:2781] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:08.243904 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80c93e207b64e01bd62338e133da1d4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:08.243961 4551 raft_consensus.cc:482] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:08.243993 4551 raft_consensus.cc:3037] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.244022 29215 raft_consensus.cc:2454] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:08.244716 4551 raft_consensus.cc:504] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.245085 4551 leader_election.cc:288] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:08.245369 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80c93e207b64e01bd62338e133da1d4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:08.245487 29215 raft_consensus.cc:3037] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.246582 29215 raft_consensus.cc:2454] T b80c93e207b64e01bd62338e133da1d4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:08.247205 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a5f8bfeddd1443da789f67949830f99 with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:08.247313 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80c93e207b64e01bd62338e133da1d4" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:08.247419 27064 raft_consensus.cc:3037] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.247624 5384 tablet_replica.cc:330] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:08.247805 5384 raft_consensus.cc:2229] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:08.247989 5384 raft_consensus.cc:2258] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:08.248369 5384 ts_tablet_manager.cc:1875] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:08.248378 27064 raft_consensus.cc:2454] T b80c93e207b64e01bd62338e133da1d4 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:08.248641 29285 leader_election.cc:302] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:59:08.248934 4551 raft_consensus.cc:2781] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:08.249017 4551 raft_consensus.cc:686] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:08.249126 4551 consensus_queue.cc:237] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.250713 5384 ts_tablet_manager.cc:1888] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:08.251048 5384 log.cc:1196] T 6a5f8bfeddd1443da789f67949830f99 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/6a5f8bfeddd1443da789f67949830f99
I20240502 01:59:08.251803 26507 catalog_manager.cc:5484] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:08.252339 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 6a5f8bfeddd1443da789f67949830f99 (table rebalancer_extra_table_20 [id=eb586a77305a4629abb1ab153d16db2b]) successfully deleted
I20240502 01:59:08.295467 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:08.300882 26794 raft_consensus.cc:1261] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:08.302337 4649 consensus_queue.cc:1035] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.304873 29487 raft_consensus.cc:1261] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:08.305472 4649 consensus_queue.cc:1035] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.309408 26509 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]). Expected version 1 got 0
I20240502 01:59:08.310169 26514 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]). Expected version 1 got 0
I20240502 01:59:08.315550 26514 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]). Expected version 1 got 0
I20240502 01:59:08.317229 29079 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:08.317701 5427 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:08.317795 4734 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.317983 5429 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:08.318578 5429 tablet.cc:1620] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:08.318923 5427 tablet.cc:1620] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:08.321920 5428 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:08.320241 26514 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c]). Expected version 1 got 0
I20240502 01:59:08.322768 5428 tablet.cc:1620] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:08.329697 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c] alter complete (version 1)
I20240502 01:59:08.377820 27044 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:08.378615 26659 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:08.381167 4734 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.381747 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.383598 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:08.395957 26641 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:08.405967 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:08.420552 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:08.422909 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.433152 29215 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:08.433951 5322 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.436493 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:08.441007 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:08.444753 5427 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:08.445071 5427 tablet.cc:1620] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:08.445351 5429 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:08.445595 5429 tablet.cc:1620] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:08.445828 5428 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:08.446110 5428 tablet.cc:1620] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:08.450449 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=9f5da10b78814ae3aace9e11209ac47c] alter complete (version 2)
I20240502 01:59:08.477555 27064 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3"
dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:45676
I20240502 01:59:08.477692 27064 raft_consensus.cc:595] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Received request to transfer leadership
I20240502 01:59:08.489033 3791 raft_consensus.cc:979] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: : Instructing follower 998b57e2badc4d779c586c6579149114 to start an election
I20240502 01:59:08.489130 3791 raft_consensus.cc:1067] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Signalling peer 998b57e2badc4d779c586c6579149114 to start an election
I20240502 01:59:08.489605 26659 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3"
dest_uuid: "998b57e2badc4d779c586c6579149114"
from {username='slave'} at 127.19.43.196:54901
I20240502 01:59:08.489760 26659 raft_consensus.cc:482] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:08.489835 26659 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:08.490813 26659 raft_consensus.cc:504] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:08.491225 26659 leader_election.cc:288] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 2 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:08.491469 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:08.491580 26927 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:08.491552 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:08.491652 29487 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:08.492486 26927 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 2.
I20240502 01:59:08.492472 29487 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 2.
I20240502 01:59:08.492884 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b546a99821624f0588a4c0f59a5ec5a3" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:08.492990 27064 raft_consensus.cc:3032] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:08.493029 27064 raft_consensus.cc:729] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:08.493094 26595 leader_election.cc:302] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:08.493173 27064 consensus_queue.cc:260] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:08.493243 3306 raft_consensus.cc:2781] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:08.493321 3306 raft_consensus.cc:686] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 2 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:08.493372 27064 raft_consensus.cc:3037] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:08.493440 3306 consensus_queue.cc:237] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:08.494412 27064 raft_consensus.cc:2454] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 2.
I20240502 01:59:08.496076 26509 catalog_manager.cc:5484] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 1 to 2, leader changed from 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 2 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:08.525318 3306 consensus_queue.cc:1035] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.529045 3306 consensus_queue.cc:1035] T 7cd36fd57d0646389b4db76561aaf500 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.559244 26512 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:08.565321 5427 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:08.569108 5428 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:08.573411 5429 tablet.cc:1722] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:08.576689 26508 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=9f5da10b78814ae3aace9e11209ac47c] alter complete (version 3)
I20240502 01:59:08.611155 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:08.615114 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.615345 26641 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.615900 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:08.617767 5151 consensus_queue.cc:1035] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.618425 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:08.624442 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:08.624541 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.627694 29454 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:08.628029 5344 consensus_queue.cc:1035] T 57c7ef32ce164bcc9976a1a6177df0de P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.681489 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:08.681659 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:08.684540 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet a9b2529e62b3426698b9e17a1ea472fc
I20240502 01:59:08.685133 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet a9b2529e62b3426698b9e17a1ea472fc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:08 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:08.685277 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet a9b2529e62b3426698b9e17a1ea472fc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:08 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:08.685423 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet a9b2529e62b3426698b9e17a1ea472fc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:08 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:08.685586 5507 tablet_replica.cc:330] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:08.685613 5384 tablet_replica.cc:330] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:08.685729 5507 raft_consensus.cc:2229] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:08.685758 5384 raft_consensus.cc:2229] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:08.685869 5384 raft_consensus.cc:2258] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:08.685868 5506 tablet_replica.cc:330] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:08.685935 5507 raft_consensus.cc:2258] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:08.686015 5506 raft_consensus.cc:2229] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:08.686483 5506 raft_consensus.cc:2258] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:08.686636 5384 ts_tablet_manager.cc:1875] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:08.686666 5507 ts_tablet_manager.cc:1875] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:08.687103 5506 ts_tablet_manager.cc:1875] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:08.688903 5507 ts_tablet_manager.cc:1888] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:08.688974 5507 log.cc:1196] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/a9b2529e62b3426698b9e17a1ea472fc
I20240502 01:59:08.689226 5507 ts_tablet_manager.cc:1909] T a9b2529e62b3426698b9e17a1ea472fc P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:59:08.689441 5506 ts_tablet_manager.cc:1888] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:08.689519 5506 log.cc:1196] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/a9b2529e62b3426698b9e17a1ea472fc
I20240502 01:59:08.689844 5506 ts_tablet_manager.cc:1909] T a9b2529e62b3426698b9e17a1ea472fc P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:08.689853 5384 ts_tablet_manager.cc:1888] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:08.689913 5384 log.cc:1196] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/a9b2529e62b3426698b9e17a1ea472fc
I20240502 01:59:08.690131 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table_new_name [id=9f5da10b78814ae3aace9e11209ac47c]) successfully deleted
I20240502 01:59:08.690187 5384 ts_tablet_manager.cc:1909] T a9b2529e62b3426698b9e17a1ea472fc P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:08.690613 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table_new_name [id=9f5da10b78814ae3aace9e11209ac47c]) successfully deleted
I20240502 01:59:08.690873 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet a9b2529e62b3426698b9e17a1ea472fc (table rebalancer_dynamic_table_new_name [id=9f5da10b78814ae3aace9e11209ac47c]) successfully deleted
I20240502 01:59:08.783809 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:08.787559 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:08.794479 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 9b1810e20ca44a0ca05168ff94dcd962 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.794657 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 9b1810e20ca44a0ca05168ff94dcd962 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.794771 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b1810e20ca44a0ca05168ff94dcd962. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.794914 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b1810e20ca44a0ca05168ff94dcd962. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.795655 29454 tablet_service.cc:1467] Processing CreateTablet for tablet f5d9713e696e4aabb13f7009bf9ca490 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.795897 29468 tablet_service.cc:1467] Processing CreateTablet for tablet c4a41f32520b48d7895e31ca43697f99 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.797214 29195 tablet_service.cc:1467] Processing CreateTablet for tablet c4a41f32520b48d7895e31ca43697f99 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.797353 26774 tablet_service.cc:1467] Processing CreateTablet for tablet f5d9713e696e4aabb13f7009bf9ca490 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.797442 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a41f32520b48d7895e31ca43697f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.797616 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5d9713e696e4aabb13f7009bf9ca490. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.795913 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5d9713e696e4aabb13f7009bf9ca490. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.801198 5563 tablet_bootstrap.cc:492] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:08.802003 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a41f32520b48d7895e31ca43697f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.803200 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:08.803546 5561 tablet_bootstrap.cc:492] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:08.804419 26641 tablet_service.cc:1467] Processing CreateTablet for tablet c4a41f32520b48d7895e31ca43697f99 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.804653 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a41f32520b48d7895e31ca43697f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.805509 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 9b1810e20ca44a0ca05168ff94dcd962 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.805742 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b1810e20ca44a0ca05168ff94dcd962. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.805795 5561 tablet_bootstrap.cc:654] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.808495 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:08.809885 5344 consensus_queue.cc:1035] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.811985 26908 tablet_service.cc:1467] Processing CreateTablet for tablet f5d9713e696e4aabb13f7009bf9ca490 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=1855ae028a1c45e1b6c0eb0415840ed7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:08.812232 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5d9713e696e4aabb13f7009bf9ca490. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.814491 5565 tablet_bootstrap.cc:492] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:08.814589 29329 tablet_service.cc:1467] Processing CreateTablet for tablet c188c141a50a440db0b93b29648cebd4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]), partition=
I20240502 01:59:08.814792 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c188c141a50a440db0b93b29648cebd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.815421 5565 tablet_bootstrap.cc:654] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.816727 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.816784 5560 tablet_bootstrap.cc:492] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:08.817332 26774 tablet_service.cc:1467] Processing CreateTablet for tablet c188c141a50a440db0b93b29648cebd4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]), partition=
I20240502 01:59:08.817546 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c188c141a50a440db0b93b29648cebd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.818966 5151 consensus_queue.cc:1035] T b80c93e207b64e01bd62338e133da1d4 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:08.823158 5562 tablet_bootstrap.cc:492] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:08.824051 5562 tablet_bootstrap.cc:654] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.828903 5562 tablet_bootstrap.cc:492] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:08.829058 5562 ts_tablet_manager.cc:1367] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:08.832445 26640 tablet_service.cc:1467] Processing CreateTablet for tablet c188c141a50a440db0b93b29648cebd4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]), partition=
I20240502 01:59:08.832638 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c188c141a50a440db0b93b29648cebd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:08.833310 5560 tablet_bootstrap.cc:654] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.833655 26641 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:08.839392 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:08.850170 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:08.850461 5561 tablet_bootstrap.cc:492] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:08.851315 5561 ts_tablet_manager.cc:1367] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20240502 01:59:08.852520 5563 tablet_bootstrap.cc:654] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.853183 5561 raft_consensus.cc:348] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.853686 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:08.857926 5560 tablet_bootstrap.cc:492] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:08.858086 5560 ts_tablet_manager.cc:1367] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.001s
I20240502 01:59:08.859282 5560 raft_consensus.cc:348] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.860374 5560 raft_consensus.cc:374] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.860463 5560 raft_consensus.cc:729] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.860654 5560 consensus_queue.cc:260] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.861332 5562 raft_consensus.cc:348] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.861765 5560 ts_tablet_manager.cc:1398] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:59:08.861986 5560 tablet_bootstrap.cc:492] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:08.862737 5562 raft_consensus.cc:374] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.862869 5560 tablet_bootstrap.cc:654] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.862874 5562 raft_consensus.cc:729] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.863011 5562 consensus_queue.cc:260] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.864014 5560 tablet_bootstrap.cc:492] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:08.864094 5560 ts_tablet_manager.cc:1367] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:59:08.864529 5560 raft_consensus.cc:348] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.864636 5560 raft_consensus.cc:374] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.864662 5560 raft_consensus.cc:729] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.864768 5560 consensus_queue.cc:260] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.865018 5560 ts_tablet_manager.cc:1398] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:59:08.865278 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:08.871448 5561 raft_consensus.cc:374] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.871551 5561 raft_consensus.cc:729] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.871708 5561 consensus_queue.cc:260] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.872177 5562 ts_tablet_manager.cc:1398] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:59:08.873158 5561 ts_tablet_manager.cc:1398] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.022s user 0.001s sys 0.000s
I20240502 01:59:08.874264 5561 tablet_bootstrap.cc:492] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:08.875644 5570 tablet_bootstrap.cc:492] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:08.879137 29487 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:08.879746 4649 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.888947 5562 tablet_bootstrap.cc:492] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:08.889577 5570 tablet_bootstrap.cc:654] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.889806 5562 tablet_bootstrap.cc:654] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.876588 5561 tablet_bootstrap.cc:654] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.892863 5563 tablet_bootstrap.cc:492] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:08.892956 5563 ts_tablet_manager.cc:1367] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.000s
I20240502 01:59:08.893395 5563 raft_consensus.cc:348] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.893493 5563 raft_consensus.cc:374] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.893524 5563 raft_consensus.cc:729] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.893626 5563 consensus_queue.cc:260] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.893906 5563 ts_tablet_manager.cc:1398] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.894102 5563 tablet_bootstrap.cc:492] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:08.894801 5565 tablet_bootstrap.cc:492] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:08.894887 5565 ts_tablet_manager.cc:1367] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.000s
I20240502 01:59:08.894897 5563 tablet_bootstrap.cc:654] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.894903 5570 tablet_bootstrap.cc:492] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:08.895375 5565 raft_consensus.cc:348] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:08.895483 5565 raft_consensus.cc:374] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.894975 5570 ts_tablet_manager.cc:1367] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 01:59:08.895514 5565 raft_consensus.cc:729] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.895633 5565 consensus_queue.cc:260] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:08.895925 5565 ts_tablet_manager.cc:1398] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.895962 5570 raft_consensus.cc:348] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.896059 5570 raft_consensus.cc:374] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.896080 5570 raft_consensus.cc:729] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.896080 5565 tablet_bootstrap.cc:492] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:08.896966 5565 tablet_bootstrap.cc:654] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.898348 5565 tablet_bootstrap.cc:492] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:08.898414 5565 ts_tablet_manager.cc:1367] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:08.898908 5565 raft_consensus.cc:348] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.899009 5565 raft_consensus.cc:374] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.899034 5565 raft_consensus.cc:729] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.899120 5565 consensus_queue.cc:260] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.899346 5565 ts_tablet_manager.cc:1398] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.899492 5565 tablet_bootstrap.cc:492] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:08.900339 5565 tablet_bootstrap.cc:654] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:08.900892 5570 consensus_queue.cc:260] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.901360 5570 ts_tablet_manager.cc:1398] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:08.902001 5561 tablet_bootstrap.cc:492] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:08.902107 5561 ts_tablet_manager.cc:1367] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240502 01:59:08.902540 5561 raft_consensus.cc:348] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:08.902688 5561 raft_consensus.cc:374] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.902765 5561 raft_consensus.cc:729] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.902892 5561 consensus_queue.cc:260] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:08.903360 5561 ts_tablet_manager.cc:1398] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.912911 5562 tablet_bootstrap.cc:492] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:08.913046 5562 ts_tablet_manager.cc:1367] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.001s
I20240502 01:59:08.913657 5562 raft_consensus.cc:348] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.913758 5562 raft_consensus.cc:374] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.913836 5562 raft_consensus.cc:729] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.913956 5562 consensus_queue.cc:260] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.915282 5562 ts_tablet_manager.cc:1398] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:08.916208 5563 tablet_bootstrap.cc:492] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:08.916289 5563 ts_tablet_manager.cc:1367] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 01:59:08.916994 5563 raft_consensus.cc:348] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:08.917104 5563 raft_consensus.cc:374] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.917133 5563 raft_consensus.cc:729] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.917135 5565 tablet_bootstrap.cc:492] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:08.917207 5565 ts_tablet_manager.cc:1367] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240502 01:59:08.917238 5563 consensus_queue.cc:260] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:08.917491 5563 ts_tablet_manager.cc:1398] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:08.918396 26930 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:08.918828 3306 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.919623 5565 raft_consensus.cc:348] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.919723 5565 raft_consensus.cc:374] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:08.919754 5565 raft_consensus.cc:729] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:08.919888 5565 consensus_queue.cc:260] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.922816 5565 ts_tablet_manager.cc:1398] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:08.928035 5375 raft_consensus.cc:482] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:08.928117 5375 raft_consensus.cc:504] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.928501 5375 leader_election.cc:288] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:08.929404 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:08.929517 26794 raft_consensus.cc:2454] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:08.929883 29421 leader_election.cc:302] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:08.929873 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:08.929986 26927 raft_consensus.cc:2454] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:08.930301 27064 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:08.930800 5587 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:08.931630 5375 raft_consensus.cc:2781] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:08.931685 5375 raft_consensus.cc:482] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:08.931718 5375 raft_consensus.cc:3037] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.932719 5375 raft_consensus.cc:504] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.933033 5375 leader_election.cc:288] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:08.933425 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:08.933528 26929 raft_consensus.cc:3037] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.933511 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:08.933604 26794 raft_consensus.cc:3037] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.934475 26929 raft_consensus.cc:2454] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:08.934567 26794 raft_consensus.cc:2454] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:08.934999 29421 leader_election.cc:302] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:08.935221 5375 raft_consensus.cc:2781] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:08.935388 5375 raft_consensus.cc:686] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:08.935513 5375 consensus_queue.cc:237] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:08.939218 26507 catalog_manager.cc:5484] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:08.957700 27064 consensus_queue.cc:237] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:08.958945 29215 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:08.959117 29350 raft_consensus.cc:1261] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:08.959380 3791 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:08.959586 4345 consensus_queue.cc:1035] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:08.961377 3791 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:08.961743 29350 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:08.964098 26507 catalog_manager.cc:5484] T a0d96b487a964b90b720d8977a96fc5b P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:08.967078 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d96b487a964b90b720d8977a96fc5b with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:08.967319 5506 tablet_replica.cc:330] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:08.967442 5506 raft_consensus.cc:2229] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:08.967610 5506 raft_consensus.cc:2258] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:08.968122 5506 ts_tablet_manager.cc:1875] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:08.968184 29215 raft_consensus.cc:2932] T a0d96b487a964b90b720d8977a96fc5b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:08.970130 5587 raft_consensus.cc:482] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:08.970214 5587 raft_consensus.cc:504] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.970558 5587 leader_election.cc:288] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:08.970111 5506 ts_tablet_manager.cc:1888] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 01:59:08.970829 5506 log.cc:1196] T a0d96b487a964b90b720d8977a96fc5b P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/a0d96b487a964b90b720d8977a96fc5b
I20240502 01:59:08.970791 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c188c141a50a440db0b93b29648cebd4" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:08.970908 26794 raft_consensus.cc:2454] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:08.970901 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c188c141a50a440db0b93b29648cebd4" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:08.970999 29350 raft_consensus.cc:2454] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:08.971199 26594 leader_election.cc:302] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:08.971395 5587 raft_consensus.cc:2781] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:08.971444 5587 raft_consensus.cc:482] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:08.971478 5587 raft_consensus.cc:3037] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.972167 5587 raft_consensus.cc:504] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.972460 5587 leader_election.cc:288] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:08.972600 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet a0d96b487a964b90b720d8977a96fc5b (table rebalance_test_table_15 [id=ad62d3bfb6624574ba0b8907ab3885bb]) successfully deleted
I20240502 01:59:08.972815 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c188c141a50a440db0b93b29648cebd4" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:08.972884 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c188c141a50a440db0b93b29648cebd4" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:08.972913 29350 raft_consensus.cc:3037] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.972967 26794 raft_consensus.cc:3037] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:08.973796 26794 raft_consensus.cc:2454] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:08.973843 29350 raft_consensus.cc:2454] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:08.974095 26592 leader_election.cc:302] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:08.976047 5587 raft_consensus.cc:2781] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:08.976140 5587 raft_consensus.cc:686] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:08.976258 5587 consensus_queue.cc:237] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:08.978376 26511 catalog_manager.cc:5484] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:09.023200 4649 raft_consensus.cc:482] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:09.023298 4649 raft_consensus.cc:504] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:09.023635 4649 leader_election.cc:288] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:09.024012 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1810e20ca44a0ca05168ff94dcd962" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:09.024138 29350 raft_consensus.cc:2454] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:09.024457 26592 leader_election.cc:302] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:09.024464 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1810e20ca44a0ca05168ff94dcd962" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:09.024588 29213 raft_consensus.cc:2454] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:09.024617 4649 raft_consensus.cc:2781] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:09.024667 4649 raft_consensus.cc:482] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:09.024706 4649 raft_consensus.cc:3037] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.025668 4649 raft_consensus.cc:504] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:09.026013 4649 leader_election.cc:288] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:09.027783 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1810e20ca44a0ca05168ff94dcd962" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:09.027887 29350 raft_consensus.cc:3037] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.028807 29350 raft_consensus.cc:2454] T 9b1810e20ca44a0ca05168ff94dcd962 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:09.029217 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1810e20ca44a0ca05168ff94dcd962" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:09.029328 29213 raft_consensus.cc:3037] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.030329 29213 raft_consensus.cc:2454] T 9b1810e20ca44a0ca05168ff94dcd962 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:09.031059 26592 leader_election.cc:302] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:09.031757 4649 raft_consensus.cc:2781] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:09.031837 4649 raft_consensus.cc:686] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:09.031962 4649 consensus_queue.cc:237] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:09.034549 26509 catalog_manager.cc:5484] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:09.084663 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:09.088868 5587 raft_consensus.cc:482] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:09.088961 5587 raft_consensus.cc:504] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.089169 26794 raft_consensus.cc:1261] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:09.089309 5587 leader_election.cc:288] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:09.089313 29350 raft_consensus.cc:1261] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:09.089776 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a41f32520b48d7895e31ca43697f99" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:09.089918 29487 raft_consensus.cc:2454] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:09.090204 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a41f32520b48d7895e31ca43697f99" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:09.090234 26595 leader_election.cc:302] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:09.090335 29213 raft_consensus.cc:2454] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:09.090389 5587 raft_consensus.cc:2781] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:09.090436 5587 raft_consensus.cc:482] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:09.090469 5587 raft_consensus.cc:3037] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.090687 4649 consensus_queue.cc:1035] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.090896 3306 consensus_queue.cc:1035] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.091564 5587 raft_consensus.cc:504] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.091894 5587 leader_election.cc:288] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:09.092624 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a41f32520b48d7895e31ca43697f99" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:09.092753 29487 raft_consensus.cc:3037] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.093263 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a41f32520b48d7895e31ca43697f99" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:09.093375 29213 raft_consensus.cc:3037] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.093706 29487 raft_consensus.cc:2454] T c4a41f32520b48d7895e31ca43697f99 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:09.094229 26595 leader_election.cc:302] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:09.094416 29213 raft_consensus.cc:2454] T c4a41f32520b48d7895e31ca43697f99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:09.095108 26506 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]). Expected version 1 got 0
I20240502 01:59:09.095476 3306 raft_consensus.cc:2781] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:09.095568 3306 raft_consensus.cc:686] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:09.095711 3306 consensus_queue.cc:237] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.095775 26507 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]). Expected version 1 got 0
I20240502 01:59:09.096992 5491 mvcc.cc:205] Tried to move back new op lower bound from 7023063650665996288 to 7023063650207637504. Current Snapshot: MvccSnapshot[applied={T|T < 7023063650665996288}]
I20240502 01:59:09.096915 5653 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:09.097476 5653 tablet.cc:1620] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:09.097959 26506 catalog_manager.cc:5484] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:09.097152 5654 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:09.099092 5654 tablet.cc:1620] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:09.099999 26508 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]). Expected version 1 got 0
I20240502 01:59:09.104204 26508 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4] alter complete (version 1)
I20240502 01:59:09.104998 5648 mvcc.cc:205] Tried to move back new op lower bound from 7023063650665996288 to 7023063650207637504. Current Snapshot: MvccSnapshot[applied={T|T < 7023063650665996288}]
I20240502 01:59:09.110440 26507 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4]). Expected version 1 got 0
I20240502 01:59:09.104936 5663 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:09.111387 5663 tablet.cc:1620] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:09.123791 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:09.124047 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:09.127285 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:09.128218 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.128240 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.139088 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:09.139706 26640 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.157491 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:09.212114 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:09.214007 26794 consensus_queue.cc:237] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.215406 26929 raft_consensus.cc:1261] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:09.215406 29213 raft_consensus.cc:1261] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:09.215773 4734 consensus_queue.cc:1035] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:09.215978 5444 consensus_queue.cc:1035] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:09.218619 4734 raft_consensus.cc:2932] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:09.220089 5653 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:09.220425 5653 tablet.cc:1620] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:09.220932 5654 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:09.221232 5654 tablet.cc:1620] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:09.222148 26929 raft_consensus.cc:2932] T d3d8471cf7654dbdab51d855ed30be18 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:09.222982 26507 catalog_manager.cc:5484] T d3d8471cf7654dbdab51d855ed30be18 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:09.224326 29213 raft_consensus.cc:2932] T d3d8471cf7654dbdab51d855ed30be18 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:09.227842 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d8471cf7654dbdab51d855ed30be18 with delete_type TABLET_DATA_TOMBSTONED (TS f3d18a019013487ab801599cfb80e4b4 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:09.228196 5685 tablet_replica.cc:330] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:09.228637 5685 raft_consensus.cc:2229] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:09.228909 26508 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=be861efb70f846049bb0b63258738bd4] alter complete (version 2)
I20240502 01:59:09.229097 5685 raft_consensus.cc:2258] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:09.229562 5663 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:09.229854 5663 tablet.cc:1620] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:09.229990 5685 ts_tablet_manager.cc:1875] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:09.232375 5685 ts_tablet_manager.cc:1888] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:09.232506 5685 log.cc:1196] T d3d8471cf7654dbdab51d855ed30be18 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/d3d8471cf7654dbdab51d855ed30be18
I20240502 01:59:09.233345 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet d3d8471cf7654dbdab51d855ed30be18 (table rebalancer_extra_table_36 [id=03c0337bceea491f8762983f90ab3e1a]) successfully deleted
I20240502 01:59:09.333463 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:09.343075 5653 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:09.347045 5654 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:09.351455 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.351635 5663 tablet.cc:1722] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:09.356024 26508 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=be861efb70f846049bb0b63258738bd4] alter complete (version 3)
I20240502 01:59:09.362215 26640 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.367624 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:09.369478 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:09.370308 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:09.372840 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:09.375568 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:09.385100 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.436225 5587 consensus_queue.cc:1035] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.438992 5587 consensus_queue.cc:1035] T 9b1810e20ca44a0ca05168ff94dcd962 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.457353 26661 consensus_queue.cc:237] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:09.458487 29213 raft_consensus.cc:1261] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:09.458809 29487 raft_consensus.cc:1261] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:09.458856 3306 consensus_queue.cc:1035] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:09.459214 3306 consensus_queue.cc:1035] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:09.460427 5587 raft_consensus.cc:2932] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:09.460822 29213 raft_consensus.cc:2932] T 213c37d37f1c4a3c8fbc48deafa9edac P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:09.461376 29487 raft_consensus.cc:2932] T 213c37d37f1c4a3c8fbc48deafa9edac P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:09.464123 26507 catalog_manager.cc:5484] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:09.464972 26515 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:09.465091 26515 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:09.467525 27044 tablet_service.cc:1514] Processing DeleteTablet for tablet 213c37d37f1c4a3c8fbc48deafa9edac with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:09.467851 5762 tablet_replica.cc:330] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:09.467975 5762 raft_consensus.cc:2229] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:09.468169 5762 raft_consensus.cc:2258] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:09.468649 5762 ts_tablet_manager.cc:1875] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:09.470119 26515 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet c188c141a50a440db0b93b29648cebd4
I20240502 01:59:09.470602 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet c188c141a50a440db0b93b29648cebd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:09 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:09.470753 5506 tablet_replica.cc:330] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:09.470883 5506 raft_consensus.cc:2229] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:09.470889 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet c188c141a50a440db0b93b29648cebd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:09 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:09.471079 5506 raft_consensus.cc:2258] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:09.471252 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet c188c141a50a440db0b93b29648cebd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:09 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:09.471648 5506 ts_tablet_manager.cc:1875] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:09.471726 5768 tablet_replica.cc:330] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:09.471853 5768 raft_consensus.cc:2229] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:09.471958 5768 raft_consensus.cc:2258] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:09.472222 5375 consensus_queue.cc:1035] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.472628 5768 ts_tablet_manager.cc:1875] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:09.473621 5506 ts_tablet_manager.cc:1888] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:09.473695 5506 log.cc:1196] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/c188c141a50a440db0b93b29648cebd4
I20240502 01:59:09.473918 5506 ts_tablet_manager.cc:1909] T c188c141a50a440db0b93b29648cebd4 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:09.473969 5762 ts_tablet_manager.cc:1888] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:09.474056 5762 log.cc:1196] T 213c37d37f1c4a3c8fbc48deafa9edac P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/213c37d37f1c4a3c8fbc48deafa9edac
I20240502 01:59:09.474537 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table_new_name [id=be861efb70f846049bb0b63258738bd4]) successfully deleted
I20240502 01:59:09.474766 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 213c37d37f1c4a3c8fbc48deafa9edac (table rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]) successfully deleted
I20240502 01:59:09.475194 5768 ts_tablet_manager.cc:1888] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:09.475360 5768 log.cc:1196] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/c188c141a50a440db0b93b29648cebd4
I20240502 01:59:09.475239 5769 tablet_replica.cc:330] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:09.475732 5769 raft_consensus.cc:2229] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:09.475864 5769 raft_consensus.cc:2258] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:09.475879 5768 ts_tablet_manager.cc:1909] T c188c141a50a440db0b93b29648cebd4 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:09.476595 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table_new_name [id=be861efb70f846049bb0b63258738bd4]) successfully deleted
I20240502 01:59:09.476691 5375 consensus_queue.cc:1035] T f5d9713e696e4aabb13f7009bf9ca490 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.477963 5769 ts_tablet_manager.cc:1875] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:09.480211 5769 ts_tablet_manager.cc:1888] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:09.480362 5769 log.cc:1196] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/c188c141a50a440db0b93b29648cebd4
I20240502 01:59:09.480674 5769 ts_tablet_manager.cc:1909] T c188c141a50a440db0b93b29648cebd4 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:59:09.481353 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet c188c141a50a440db0b93b29648cebd4 (table rebalancer_dynamic_table_new_name [id=be861efb70f846049bb0b63258738bd4]) successfully deleted
I20240502 01:59:09.571898 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:09.578317 3306 consensus_queue.cc:1035] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.581702 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 0d34f17a52fb48f682a011accc125856 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]), partition=
I20240502 01:59:09.581719 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 0d34f17a52fb48f682a011accc125856 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]), partition=
I20240502 01:59:09.581723 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 0d34f17a52fb48f682a011accc125856 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]), partition=
I20240502 01:59:09.581943 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d34f17a52fb48f682a011accc125856. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.581944 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d34f17a52fb48f682a011accc125856. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.581944 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d34f17a52fb48f682a011accc125856. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.585331 5816 tablet_bootstrap.cc:492] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:09.585165 5814 tablet_bootstrap.cc:492] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:09.587522 4649 consensus_queue.cc:1035] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.588837 5816 tablet_bootstrap.cc:654] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.590591 5814 tablet_bootstrap.cc:654] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.591665 5816 tablet_bootstrap.cc:492] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:09.591760 5815 tablet_bootstrap.cc:492] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:09.591833 5816 ts_tablet_manager.cc:1367] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:09.592320 5816 raft_consensus.cc:348] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.592530 5816 raft_consensus.cc:374] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.592610 5816 raft_consensus.cc:729] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.593091 5815 tablet_bootstrap.cc:654] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.593734 5816 consensus_queue.cc:260] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.594139 5816 ts_tablet_manager.cc:1398] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:09.594488 5815 tablet_bootstrap.cc:492] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:09.594568 5815 ts_tablet_manager.cc:1367] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:09.594929 5815 raft_consensus.cc:348] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.595017 5815 raft_consensus.cc:374] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.595046 5815 raft_consensus.cc:729] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.595161 5815 consensus_queue.cc:260] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.595422 5815 ts_tablet_manager.cc:1398] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:09.601199 5814 tablet_bootstrap.cc:492] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:09.601292 5814 ts_tablet_manager.cc:1367] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 01:59:09.601697 5814 raft_consensus.cc:348] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.601817 5814 raft_consensus.cc:374] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.601845 5814 raft_consensus.cc:729] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.601938 5814 consensus_queue.cc:260] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.602183 5814 ts_tablet_manager.cc:1398] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:09.616189 29195 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 01:59:09.616675 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:09.629241 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:09.636518 26774 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:09.637475 27044 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.640475 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:09.646569 26640 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:09.652464 29059 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:09.671038 26510 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:09.674746 5291 raft_consensus.cc:482] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:09.674844 5291 raft_consensus.cc:504] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.675207 5291 leader_election.cc:288] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:09.678232 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d34f17a52fb48f682a011accc125856" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:09.678375 29487 raft_consensus.cc:2454] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:09.678692 26862 leader_election.cc:302] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:09.678730 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d34f17a52fb48f682a011accc125856" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:09.678855 29079 raft_consensus.cc:2454] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:09.679091 5291 raft_consensus.cc:2781] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:09.679160 5291 raft_consensus.cc:482] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:09.679200 5291 raft_consensus.cc:3037] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.680264 5291 raft_consensus.cc:504] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.680572 5291 leader_election.cc:288] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:09.680853 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d34f17a52fb48f682a011accc125856" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:09.680958 29487 raft_consensus.cc:3037] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.681155 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d34f17a52fb48f682a011accc125856" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:09.681257 29079 raft_consensus.cc:3037] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.681869 29487 raft_consensus.cc:2454] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:09.682329 26862 leader_election.cc:302] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:09.682582 29079 raft_consensus.cc:2454] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:09.682793 5291 raft_consensus.cc:2781] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:09.682879 5291 raft_consensus.cc:686] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:09.683012 5291 consensus_queue.cc:237] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.684598 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 4e43471a2de04d7bbae60753ad64b834 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.684720 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 4e43471a2de04d7bbae60753ad64b834 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.684878 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e43471a2de04d7bbae60753ad64b834. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.684970 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e43471a2de04d7bbae60753ad64b834. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.685302 26773 tablet_service.cc:1467] Processing CreateTablet for tablet 42479aa7dbbb4f6f94fca940fe8ec4a5 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.685525 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42479aa7dbbb4f6f94fca940fe8ec4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.684967 26775 tablet_service.cc:1467] Processing CreateTablet for tablet e8a720d541ed46848558a7fc861db6f7 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.686615 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a720d541ed46848558a7fc861db6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.687378 26908 tablet_service.cc:1467] Processing CreateTablet for tablet e8a720d541ed46848558a7fc861db6f7 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.687546 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 4e43471a2de04d7bbae60753ad64b834 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.687613 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a720d541ed46848558a7fc861db6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.687794 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e43471a2de04d7bbae60753ad64b834. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.687860 29059 tablet_service.cc:1467] Processing CreateTablet for tablet e8a720d541ed46848558a7fc861db6f7 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.688086 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a720d541ed46848558a7fc861db6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.688203 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 42479aa7dbbb4f6f94fca940fe8ec4a5 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.688447 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42479aa7dbbb4f6f94fca940fe8ec4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.690083 5815 tablet_bootstrap.cc:492] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:09.690258 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 42479aa7dbbb4f6f94fca940fe8ec4a5 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=cd53408e1a8541a4a484786b065ee080]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:09.690464 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42479aa7dbbb4f6f94fca940fe8ec4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:09.690599 5836 tablet_bootstrap.cc:492] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:09.691058 5815 tablet_bootstrap.cc:654] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.691504 5836 tablet_bootstrap.cc:654] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.691766 26506 catalog_manager.cc:5484] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:09.692574 5836 tablet_bootstrap.cc:492] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:09.692662 5836 ts_tablet_manager.cc:1367] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:09.693085 5836 raft_consensus.cc:348] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.693177 5836 raft_consensus.cc:374] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.693207 5836 raft_consensus.cc:729] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.693318 5836 consensus_queue.cc:260] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.694368 5816 tablet_bootstrap.cc:492] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:09.695283 5816 tablet_bootstrap.cc:654] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.696009 5836 ts_tablet_manager.cc:1398] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:09.697419 5815 tablet_bootstrap.cc:492] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:09.697507 5815 ts_tablet_manager.cc:1367] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:09.697909 5815 raft_consensus.cc:348] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.697999 5815 raft_consensus.cc:374] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.698062 5815 raft_consensus.cc:729] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.698179 5815 consensus_queue.cc:260] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.698454 5815 ts_tablet_manager.cc:1398] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:09.699858 5816 tablet_bootstrap.cc:492] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:09.699937 5816 ts_tablet_manager.cc:1367] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:09.700070 5839 tablet_bootstrap.cc:492] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:09.700384 5816 raft_consensus.cc:348] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.700495 5816 raft_consensus.cc:374] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.700528 5816 raft_consensus.cc:729] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.700641 5816 consensus_queue.cc:260] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.700763 5835 tablet_bootstrap.cc:492] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:09.700918 5816 ts_tablet_manager.cc:1398] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:09.701663 5839 tablet_bootstrap.cc:654] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.702620 5835 tablet_bootstrap.cc:654] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.704413 5839 tablet_bootstrap.cc:492] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:09.704524 5839 ts_tablet_manager.cc:1367] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:09.705159 5839 raft_consensus.cc:348] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.705672 5835 tablet_bootstrap.cc:492] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:09.705758 5839 raft_consensus.cc:374] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.705839 5839 raft_consensus.cc:729] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.705956 5835 ts_tablet_manager.cc:1367] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 01:59:09.706533 5835 raft_consensus.cc:348] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.706725 5839 consensus_queue.cc:260] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.707008 5839 ts_tablet_manager.cc:1398] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:09.707162 5835 raft_consensus.cc:374] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.707460 5835 raft_consensus.cc:729] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.707589 5835 consensus_queue.cc:260] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.708580 5835 ts_tablet_manager.cc:1398] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:09.708768 5835 tablet_bootstrap.cc:492] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:09.709594 5835 tablet_bootstrap.cc:654] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.711879 5834 tablet_bootstrap.cc:492] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:09.712822 5834 tablet_bootstrap.cc:654] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.714257 5834 tablet_bootstrap.cc:492] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:09.714393 5834 ts_tablet_manager.cc:1367] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:09.714875 5834 raft_consensus.cc:348] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.715035 5834 raft_consensus.cc:374] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.715101 5835 tablet_bootstrap.cc:492] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:09.715113 5834 raft_consensus.cc:729] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.715224 5835 ts_tablet_manager.cc:1367] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:09.715327 5834 consensus_queue.cc:260] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.715672 5835 raft_consensus.cc:348] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.715795 5835 raft_consensus.cc:374] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.715839 5835 raft_consensus.cc:729] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.715950 5835 consensus_queue.cc:260] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.716214 5835 ts_tablet_manager.cc:1398] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:09.716610 5834 ts_tablet_manager.cc:1398] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:09.716876 5834 tablet_bootstrap.cc:492] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:09.717726 5834 tablet_bootstrap.cc:654] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.724162 5834 tablet_bootstrap.cc:492] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:09.724253 5834 ts_tablet_manager.cc:1367] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:59:09.724661 5834 raft_consensus.cc:348] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.724748 5834 raft_consensus.cc:374] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.724776 5834 raft_consensus.cc:729] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.724884 5834 consensus_queue.cc:260] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.725155 5834 ts_tablet_manager.cc:1398] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:09.725376 5834 tablet_bootstrap.cc:492] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:09.726142 5834 tablet_bootstrap.cc:654] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:09.727728 5834 tablet_bootstrap.cc:492] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:09.727862 5834 ts_tablet_manager.cc:1367] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:59:09.728278 5834 raft_consensus.cc:348] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.728384 5834 raft_consensus.cc:374] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:09.728415 5834 raft_consensus.cc:729] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:09.728583 5834 consensus_queue.cc:260] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.728971 5834 ts_tablet_manager.cc:1398] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:09.739156 27064 consensus_queue.cc:237] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.740442 26793 raft_consensus.cc:1261] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:09.740509 26927 raft_consensus.cc:1261] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:09.740816 5739 consensus_queue.cc:1035] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:09.741001 4345 consensus_queue.cc:1035] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:09.742705 3791 raft_consensus.cc:2932] T ab7a9b176213400f990c4d5df0d7567f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:09.743177 26793 raft_consensus.cc:2932] T ab7a9b176213400f990c4d5df0d7567f P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:09.744112 26927 raft_consensus.cc:2932] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:09.746850 26510 catalog_manager.cc:5484] T ab7a9b176213400f990c4d5df0d7567f P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:09.750891 29330 tablet_service.cc:1514] Processing DeleteTablet for tablet ab7a9b176213400f990c4d5df0d7567f with delete_type TABLET_DATA_TOMBSTONED (TS 468dd27456c64d67b22e8a58af93d93d not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:09.751085 5769 tablet_replica.cc:330] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:09.751488 5769 raft_consensus.cc:2229] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:09.751618 5769 raft_consensus.cc:2258] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:09.752048 5769 ts_tablet_manager.cc:1875] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:09.753720 5769 ts_tablet_manager.cc:1888] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:09.753782 5769 log.cc:1196] T ab7a9b176213400f990c4d5df0d7567f P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/ab7a9b176213400f990c4d5df0d7567f
I20240502 01:59:09.754367 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet ab7a9b176213400f990c4d5df0d7567f (table rebalancer_extra_table_46 [id=ad29c5724e164cb7a12c4a0491705b69]) successfully deleted
I20240502 01:59:09.776067 5818 raft_consensus.cc:482] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:09.776180 5818 raft_consensus.cc:504] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.776635 5818 leader_election.cc:288] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:09.777005 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42479aa7dbbb4f6f94fca940fe8ec4a5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:09.777005 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42479aa7dbbb4f6f94fca940fe8ec4a5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:09.777122 27064 raft_consensus.cc:2454] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:09.777120 26793 raft_consensus.cc:2454] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:09.777405 29149 leader_election.cc:302] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:09.777710 5818 raft_consensus.cc:2781] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:09.777765 5818 raft_consensus.cc:482] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:09.777797 5818 raft_consensus.cc:3037] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.778828 5818 raft_consensus.cc:504] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.779150 5818 leader_election.cc:288] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:09.779764 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42479aa7dbbb4f6f94fca940fe8ec4a5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:09.779865 27063 raft_consensus.cc:3037] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.780401 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42479aa7dbbb4f6f94fca940fe8ec4a5" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:09.780503 26793 raft_consensus.cc:3037] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.780751 27063 raft_consensus.cc:2454] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:09.781126 29150 leader_election.cc:302] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:09.781483 26793 raft_consensus.cc:2454] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:09.781759 5818 raft_consensus.cc:2781] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:09.781858 5818 raft_consensus.cc:686] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:09.781991 5818 consensus_queue.cc:237] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:09.784202 26506 catalog_manager.cc:5484] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:09.797502 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:09.802742 29079 raft_consensus.cc:1261] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:09.802752 29487 raft_consensus.cc:1261] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:09.803676 5291 consensus_queue.cc:1035] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.804260 5291 consensus_queue.cc:1035] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:09.807865 26514 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]). Expected version 1 got 0
I20240502 01:59:09.812204 26514 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]). Expected version 1 got 0
I20240502 01:59:09.815901 5773 mvcc.cc:205] Tried to move back new op lower bound from 7023063653588156416 to 7023063653103038464. Current Snapshot: MvccSnapshot[applied={T|T < 7023063653588156416}]
I20240502 01:59:09.818928 26510 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]). Expected version 1 got 0
I20240502 01:59:09.822455 5912 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:09.822938 5912 tablet.cc:1620] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:09.824102 5772 mvcc.cc:205] Tried to move back new op lower bound from 7023063653588156416 to 7023063653103038464. Current Snapshot: MvccSnapshot[applied={T|T < 7023063653588156416}]
I20240502 01:59:09.824605 5914 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:09.825003 5914 tablet.cc:1620] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:09.826463 26514 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2]). Expected version 1 got 0
I20240502 01:59:09.835044 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2] alter complete (version 1)
I20240502 01:59:09.847347 5913 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:09.848347 5913 tablet.cc:1620] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:09.930866 5738 raft_consensus.cc:482] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:09.930985 5738 raft_consensus.cc:504] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.931442 5738 leader_election.cc:288] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:09.931974 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a720d541ed46848558a7fc861db6f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:09.932098 26927 raft_consensus.cc:2454] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:09.932293 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a720d541ed46848558a7fc861db6f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:09.932370 29016 leader_election.cc:302] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:09.932402 26793 raft_consensus.cc:2454] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:09.932535 5738 raft_consensus.cc:2781] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:09.932585 5738 raft_consensus.cc:482] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:09.932628 5738 raft_consensus.cc:3037] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.933773 5738 raft_consensus.cc:504] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.934113 5738 leader_election.cc:288] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:09.934412 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a720d541ed46848558a7fc861db6f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:09.934500 26927 raft_consensus.cc:3037] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.935411 26927 raft_consensus.cc:2454] T e8a720d541ed46848558a7fc861db6f7 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:09.935611 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a720d541ed46848558a7fc861db6f7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:09.935705 26793 raft_consensus.cc:3037] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.936105 29016 leader_election.cc:302] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:09.936475 5738 raft_consensus.cc:2781] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:09.936628 26793 raft_consensus.cc:2454] T e8a720d541ed46848558a7fc861db6f7 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:09.936813 5738 raft_consensus.cc:686] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:09.936949 5738 consensus_queue.cc:237] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:09.937629 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:09.940062 26512 catalog_manager.cc:5484] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:09.944274 5912 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:09.944600 5912 tablet.cc:1620] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:09.945173 5914 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:09.945452 5914 tablet.cc:1620] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:09.947945 5913 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:09.948204 5913 tablet.cc:1620] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:09.950232 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=c279fbc3603f42058dd359768f9e2ab2] alter complete (version 2)
I20240502 01:59:09.983841 4345 raft_consensus.cc:482] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:09.983932 4345 raft_consensus.cc:504] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.984293 4345 leader_election.cc:288] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:09.984540 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e43471a2de04d7bbae60753ad64b834" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:09.984644 26793 raft_consensus.cc:2454] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:09.984643 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e43471a2de04d7bbae60753ad64b834" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:09.984776 26661 raft_consensus.cc:2454] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:09.985100 26997 leader_election.cc:302] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:09.985268 4345 raft_consensus.cc:2781] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:09.985329 4345 raft_consensus.cc:482] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:09.985363 4345 raft_consensus.cc:3037] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.986408 4345 raft_consensus.cc:504] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.986729 4345 leader_election.cc:288] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:09.990468 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e43471a2de04d7bbae60753ad64b834" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:09.990597 26793 raft_consensus.cc:3037] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.991070 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e43471a2de04d7bbae60753ad64b834" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:09.991176 26661 raft_consensus.cc:3037] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:09.991557 26793 raft_consensus.cc:2454] T 4e43471a2de04d7bbae60753ad64b834 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:09.991824 26997 leader_election.cc:302] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:09.992019 4345 raft_consensus.cc:2781] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:09.992084 4345 raft_consensus.cc:686] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:09.992138 26661 raft_consensus.cc:2454] T 4e43471a2de04d7bbae60753ad64b834 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:09.992195 4345 consensus_queue.cc:237] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:09.994433 26509 catalog_manager.cc:5484] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:10.020728 26640 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:10.027204 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:10.034948 29195 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:10.035171 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.036204 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:10.036558 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:10.040597 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.041749 27043 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:10.058785 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:10.064851 5912 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:10.067919 5914 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:10.067936 5913 tablet.cc:1722] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:10.073807 26514 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=c279fbc3603f42058dd359768f9e2ab2] alter complete (version 3)
I20240502 01:59:10.117710 26927 consensus_queue.cc:237] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:10.118790 29213 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:10.119172 5917 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:10.119325 29080 raft_consensus.cc:1261] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:10.119674 5917 consensus_queue.cc:1035] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:10.121399 5291 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:10.121834 29080 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:10.122525 29213 raft_consensus.cc:2932] T 5515469fdf494900ae20f20fdfca587c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:10.125272 26509 catalog_manager.cc:5484] T 5515469fdf494900ae20f20fdfca587c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 6 to 7, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:10.129408 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 5515469fdf494900ae20f20fdfca587c with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:10.130331 5954 tablet_replica.cc:330] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:10.130482 5954 raft_consensus.cc:2229] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:10.130856 5954 raft_consensus.cc:2258] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.131505 5954 ts_tablet_manager.cc:1875] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:10.133829 5954 ts_tablet_manager.cc:1888] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 01:59:10.133893 5954 log.cc:1196] T 5515469fdf494900ae20f20fdfca587c P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/5515469fdf494900ae20f20fdfca587c
I20240502 01:59:10.134375 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 5515469fdf494900ae20f20fdfca587c (table rebalance_test_table_1 [id=24546c62f4d34ec5a528011afd24c010]) successfully deleted
I20240502 01:59:10.177732 26508 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:10.177865 26508 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:10.180255 26508 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 0d34f17a52fb48f682a011accc125856
I20240502 01:59:10.180907 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d34f17a52fb48f682a011accc125856 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:10 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:10.180935 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d34f17a52fb48f682a011accc125856 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:10 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:10.180948 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d34f17a52fb48f682a011accc125856 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:10 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:10.181224 5992 tablet_replica.cc:330] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:10.181335 5993 tablet_replica.cc:330] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:10.181412 5992 raft_consensus.cc:2229] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:10.181460 5993 raft_consensus.cc:2229] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:10.181547 5992 raft_consensus.cc:2258] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.181654 5993 raft_consensus.cc:2258] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.182096 5992 ts_tablet_manager.cc:1875] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:10.182298 5993 ts_tablet_manager.cc:1875] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:10.184176 5992 ts_tablet_manager.cc:1888] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:10.184252 5992 log.cc:1196] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/0d34f17a52fb48f682a011accc125856
I20240502 01:59:10.184509 5992 ts_tablet_manager.cc:1909] T 0d34f17a52fb48f682a011accc125856 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:59:10.184515 5993 ts_tablet_manager.cc:1888] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:10.184582 5993 log.cc:1196] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/0d34f17a52fb48f682a011accc125856
I20240502 01:59:10.184875 5993 ts_tablet_manager.cc:1909] T 0d34f17a52fb48f682a011accc125856 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:59:10.186125 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table_new_name [id=c279fbc3603f42058dd359768f9e2ab2]) successfully deleted
I20240502 01:59:10.186595 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table_new_name [id=c279fbc3603f42058dd359768f9e2ab2]) successfully deleted
I20240502 01:59:10.186709 5994 tablet_replica.cc:330] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:10.186852 5994 raft_consensus.cc:2229] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:10.186951 5994 raft_consensus.cc:2258] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.187420 5994 ts_tablet_manager.cc:1875] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:10.189255 5994 ts_tablet_manager.cc:1888] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:10.189522 5994 log.cc:1196] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/0d34f17a52fb48f682a011accc125856
I20240502 01:59:10.189810 5994 ts_tablet_manager.cc:1909] T 0d34f17a52fb48f682a011accc125856 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:10.192159 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 0d34f17a52fb48f682a011accc125856 (table rebalancer_dynamic_table_new_name [id=c279fbc3603f42058dd359768f9e2ab2]) successfully deleted
I20240502 01:59:10.248687 29195 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:10.253082 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.259234 27043 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:10.259485 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:10.263242 26640 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:10.264308 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:10.269534 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.281937 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:10.288291 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb0fda261894d3c8372d5e3c15164eb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]), partition=
I20240502 01:59:10.288529 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb0fda261894d3c8372d5e3c15164eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.292382 6006 tablet_bootstrap.cc:492] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:10.294790 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb0fda261894d3c8372d5e3c15164eb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]), partition=
I20240502 01:59:10.295007 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb0fda261894d3c8372d5e3c15164eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.297288 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb0fda261894d3c8372d5e3c15164eb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]), partition=
I20240502 01:59:10.297489 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb0fda261894d3c8372d5e3c15164eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.299554 6007 tablet_bootstrap.cc:492] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:10.300647 6007 tablet_bootstrap.cc:654] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.301664 6007 tablet_bootstrap.cc:492] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:10.301740 6007 ts_tablet_manager.cc:1367] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:10.302189 6007 raft_consensus.cc:348] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.302291 6007 raft_consensus.cc:374] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.302320 6007 raft_consensus.cc:729] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.302430 6007 consensus_queue.cc:260] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.302683 6007 ts_tablet_manager.cc:1398] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:10.303985 6008 tablet_bootstrap.cc:492] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:10.304911 6008 tablet_bootstrap.cc:654] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.305990 6008 tablet_bootstrap.cc:492] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:10.306083 6008 ts_tablet_manager.cc:1367] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:10.306461 6008 raft_consensus.cc:348] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.306545 6008 raft_consensus.cc:374] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.306571 6008 raft_consensus.cc:729] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.306710 6008 consensus_queue.cc:260] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.306969 6008 ts_tablet_manager.cc:1398] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:10.307533 6006 tablet_bootstrap.cc:654] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.308805 6006 tablet_bootstrap.cc:492] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:10.308888 6006 ts_tablet_manager.cc:1367] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 01:59:10.310130 6006 raft_consensus.cc:348] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.310220 6006 raft_consensus.cc:374] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.310241 6006 raft_consensus.cc:729] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.310338 6006 consensus_queue.cc:260] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.310603 6006 ts_tablet_manager.cc:1398] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:10.313650 6012 consensus_queue.cc:1035] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.316555 5818 consensus_queue.cc:1035] T 42479aa7dbbb4f6f94fca940fe8ec4a5 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.329689 26908 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:10.370209 26661 consensus_queue.cc:237] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.371472 29487 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:10.371501 29080 raft_consensus.cc:1261] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:10.371901 6034 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:10.372853 4649 consensus_queue.cc:1035] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:10.374527 4649 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:10.375285 29080 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:10.377950 26511 catalog_manager.cc:5484] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:10.378576 29487 raft_consensus.cc:2932] T e1cd4681de8645ff92b479ccdd3eac37 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:10.379040 3791 consensus_queue.cc:1035] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.382856 4345 consensus_queue.cc:1035] T 4e43471a2de04d7bbae60753ad64b834 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.385938 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet e1cd4681de8645ff92b479ccdd3eac37 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:10.386154 5993 tablet_replica.cc:330] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:10.386298 5993 raft_consensus.cc:2229] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:10.386438 5993 raft_consensus.cc:2258] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.386969 5993 ts_tablet_manager.cc:1875] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:10.388962 5993 ts_tablet_manager.cc:1888] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 01:59:10.389024 5993 log.cc:1196] T e1cd4681de8645ff92b479ccdd3eac37 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/e1cd4681de8645ff92b479ccdd3eac37
I20240502 01:59:10.389655 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet e1cd4681de8645ff92b479ccdd3eac37 (table rebalance_test_table_11 [id=3a48f5c202c04fe8bc5c1901bd327d46]) successfully deleted
I20240502 01:59:10.434815 5443 consensus_queue.cc:1035] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.439603 5738 consensus_queue.cc:1035] T e8a720d541ed46848558a7fc861db6f7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.450301 5917 raft_consensus.cc:482] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:10.450398 5917 raft_consensus.cc:504] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.450872 5917 leader_election.cc:288] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:10.451164 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb0fda261894d3c8372d5e3c15164eb" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:10.451292 27063 raft_consensus.cc:2454] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:10.451318 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb0fda261894d3c8372d5e3c15164eb" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:10.451442 26661 raft_consensus.cc:2454] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:10.451546 26862 leader_election.cc:302] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:10.451692 5917 raft_consensus.cc:2781] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:10.451745 5917 raft_consensus.cc:482] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:10.451781 5917 raft_consensus.cc:3037] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.452831 5917 raft_consensus.cc:504] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.453158 5917 leader_election.cc:288] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:10.453390 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb0fda261894d3c8372d5e3c15164eb" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:10.453498 26661 raft_consensus.cc:3037] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.453572 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb0fda261894d3c8372d5e3c15164eb" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:10.453680 27063 raft_consensus.cc:3037] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.454542 26661 raft_consensus.cc:2454] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:10.454710 27063 raft_consensus.cc:2454] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:10.454923 26861 leader_election.cc:302] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:10.455752 5917 raft_consensus.cc:2781] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:10.455835 5917 raft_consensus.cc:686] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:10.455953 5917 consensus_queue.cc:237] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.459369 26511 catalog_manager.cc:5484] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:10.512701 29330 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.513414 27043 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:10.516912 29195 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 01:59:10.526517 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:10.528268 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:10.540028 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:10.553311 26513 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:10.554592 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.557593 26640 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:10.563217 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 00d8c5f39ce544018d3f1aa9138e268a (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.563479 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d8c5f39ce544018d3f1aa9138e268a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.563719 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 1fba57dcf4aa4bcf964556f707d8dc99 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.563984 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fba57dcf4aa4bcf964556f707d8dc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.566159 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 00d8c5f39ce544018d3f1aa9138e268a (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.566270 29330 tablet_service.cc:1467] Processing CreateTablet for tablet 00d8c5f39ce544018d3f1aa9138e268a (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.566413 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d8c5f39ce544018d3f1aa9138e268a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.566515 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d8c5f39ce544018d3f1aa9138e268a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.566561 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 1fba57dcf4aa4bcf964556f707d8dc99 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.566797 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fba57dcf4aa4bcf964556f707d8dc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.568718 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 67686979ffb54564986fe0733f57a8d2 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.568936 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67686979ffb54564986fe0733f57a8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.570245 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:10.570520 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 1fba57dcf4aa4bcf964556f707d8dc99 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.570780 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fba57dcf4aa4bcf964556f707d8dc99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.571192 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 67686979ffb54564986fe0733f57a8d2 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.571278 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 67686979ffb54564986fe0733f57a8d2 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=41d280809a244969a6069a2ac5b80e86]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:10.571413 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67686979ffb54564986fe0733f57a8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.571486 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67686979ffb54564986fe0733f57a8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:10.572853 6008 tablet_bootstrap.cc:492] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:10.573715 6008 tablet_bootstrap.cc:654] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.576417 6099 tablet_bootstrap.cc:492] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:10.578599 27063 raft_consensus.cc:1261] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:10.578682 26661 raft_consensus.cc:1261] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:10.579022 6099 tablet_bootstrap.cc:654] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.579134 5917 consensus_queue.cc:1035] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.580087 6008 tablet_bootstrap.cc:492] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:10.580176 6008 ts_tablet_manager.cc:1367] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 01:59:10.580425 5917 consensus_queue.cc:1035] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:10.580578 6008 raft_consensus.cc:348] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.580677 6008 raft_consensus.cc:374] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.580705 6008 raft_consensus.cc:729] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.580806 6008 consensus_queue.cc:260] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.581055 6008 ts_tablet_manager.cc:1398] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:10.581216 6008 tablet_bootstrap.cc:492] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:10.581902 26513 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]). Expected version 1 got 0
I20240502 01:59:10.583029 6008 tablet_bootstrap.cc:654] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.582995 6103 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:10.584470 6103 tablet.cc:1620] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:10.585155 26506 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]). Expected version 1 got 0
I20240502 01:59:10.590167 26513 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]). Expected version 1 got 0
I20240502 01:59:10.596818 26506 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411] alter complete (version 1)
I20240502 01:59:10.599257 6098 tablet_bootstrap.cc:492] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:10.599418 26508 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411]). Expected version 1 got 0
I20240502 01:59:10.602959 6098 tablet_bootstrap.cc:654] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.604446 6107 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:10.604782 6107 tablet.cc:1620] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:10.604947 6096 tablet_bootstrap.cc:492] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:10.606823 6096 tablet_bootstrap.cc:654] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.607494 6097 tablet_bootstrap.cc:492] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:10.607503 6104 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:10.607800 6104 tablet.cc:1620] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:10.608537 6008 tablet_bootstrap.cc:492] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:10.608620 6008 ts_tablet_manager.cc:1367] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20240502 01:59:10.608872 6097 tablet_bootstrap.cc:654] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.609025 6008 raft_consensus.cc:348] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.609122 6008 raft_consensus.cc:374] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.609154 6008 raft_consensus.cc:729] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.609267 6008 consensus_queue.cc:260] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.609537 6008 ts_tablet_manager.cc:1398] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:10.612071 6096 tablet_bootstrap.cc:492] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:10.612401 6096 ts_tablet_manager.cc:1367] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:59:10.612885 6096 raft_consensus.cc:348] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.613600 6099 tablet_bootstrap.cc:492] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:10.613693 6099 ts_tablet_manager.cc:1367] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20240502 01:59:10.613867 6098 tablet_bootstrap.cc:492] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:10.613940 6098 ts_tablet_manager.cc:1367] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:59:10.614145 6099 raft_consensus.cc:348] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.614245 6099 raft_consensus.cc:374] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.614279 6099 raft_consensus.cc:729] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.614394 6099 consensus_queue.cc:260] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.614459 6096 raft_consensus.cc:374] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.614508 6096 raft_consensus.cc:729] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.614656 6099 ts_tablet_manager.cc:1398] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:10.614712 6098 raft_consensus.cc:348] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.614809 6098 raft_consensus.cc:374] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.614871 6098 raft_consensus.cc:729] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.615041 6098 consensus_queue.cc:260] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.615370 6098 ts_tablet_manager.cc:1398] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:10.614619 6096 consensus_queue.cc:260] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.615614 6098 tablet_bootstrap.cc:492] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:10.617179 6098 tablet_bootstrap.cc:654] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.617679 6096 ts_tablet_manager.cc:1398] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:59:10.619407 6096 tablet_bootstrap.cc:492] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:10.620008 5930 raft_consensus.cc:482] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:10.620096 5930 raft_consensus.cc:504] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.620406 5930 leader_election.cc:288] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:10.620836 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d8c5f39ce544018d3f1aa9138e268a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:10.620936 29489 raft_consensus.cc:2454] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:10.621621 29286 leader_election.cc:302] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:10.621754 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d8c5f39ce544018d3f1aa9138e268a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:10.621785 5930 raft_consensus.cc:2781] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:10.621840 5930 raft_consensus.cc:482] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:10.621870 5930 raft_consensus.cc:3037] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.621860 29080 raft_consensus.cc:2454] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:10.622880 5930 raft_consensus.cc:504] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.623150 6096 tablet_bootstrap.cc:654] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.623190 5930 leader_election.cc:288] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:10.623566 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d8c5f39ce544018d3f1aa9138e268a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:10.623656 29489 raft_consensus.cc:3037] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.624405 29489 raft_consensus.cc:2454] T 00d8c5f39ce544018d3f1aa9138e268a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:10.624660 29286 leader_election.cc:302] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:10.624817 5930 raft_consensus.cc:2781] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:10.624888 5930 raft_consensus.cc:686] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:10.625000 5930 consensus_queue.cc:237] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:10.626475 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d8c5f39ce544018d3f1aa9138e268a" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:10.626574 29080 raft_consensus.cc:3037] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.626632 6096 tablet_bootstrap.cc:492] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:10.626708 6096 ts_tablet_manager.cc:1367] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:59:10.627305 6096 raft_consensus.cc:348] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.627409 6096 raft_consensus.cc:374] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.627441 6096 raft_consensus.cc:729] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.627444 29080 raft_consensus.cc:2454] T 00d8c5f39ce544018d3f1aa9138e268a P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:10.627545 6096 consensus_queue.cc:260] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.628731 6096 ts_tablet_manager.cc:1398] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:10.629287 26513 catalog_manager.cc:5484] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:10.637254 6097 tablet_bootstrap.cc:492] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:10.637344 6097 ts_tablet_manager.cc:1367] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 01:59:10.637408 6098 tablet_bootstrap.cc:492] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:10.637476 6098 ts_tablet_manager.cc:1367] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240502 01:59:10.637789 6097 raft_consensus.cc:348] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.637879 6097 raft_consensus.cc:374] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.637833 6098 raft_consensus.cc:348] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.637899 6097 raft_consensus.cc:729] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.637914 6098 raft_consensus.cc:374] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.637938 6098 raft_consensus.cc:729] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.637993 6097 consensus_queue.cc:260] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.638062 6098 consensus_queue.cc:260] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.638267 6097 ts_tablet_manager.cc:1398] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:10.638330 6098 ts_tablet_manager.cc:1398] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:10.638453 6098 tablet_bootstrap.cc:492] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:10.639529 6098 tablet_bootstrap.cc:654] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:10.640873 6098 tablet_bootstrap.cc:492] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:10.640992 6098 ts_tablet_manager.cc:1367] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:59:10.641575 6098 raft_consensus.cc:348] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.641671 6098 raft_consensus.cc:374] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:10.641698 6098 raft_consensus.cc:729] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:10.641801 6098 consensus_queue.cc:260] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.642084 6098 ts_tablet_manager.cc:1398] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:10.646011 29489 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2"
dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58352
I20240502 01:59:10.646165 29489 raft_consensus.cc:595] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Received request to transfer leadership
I20240502 01:59:10.670549 5917 raft_consensus.cc:482] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:10.670653 5917 raft_consensus.cc:504] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.671052 5917 leader_election.cc:288] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:10.671375 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67686979ffb54564986fe0733f57a8d2" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:10.671489 29349 raft_consensus.cc:2454] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:10.671644 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67686979ffb54564986fe0733f57a8d2" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:10.671732 26859 leader_election.cc:302] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:10.671770 29489 raft_consensus.cc:2454] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:10.671905 5917 raft_consensus.cc:2781] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:10.671948 5917 raft_consensus.cc:482] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:10.671988 5917 raft_consensus.cc:3037] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.672956 5917 raft_consensus.cc:504] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.673255 5917 leader_election.cc:288] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:10.673501 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67686979ffb54564986fe0733f57a8d2" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:10.673539 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67686979ffb54564986fe0733f57a8d2" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:10.673609 29489 raft_consensus.cc:3037] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.673632 29349 raft_consensus.cc:3037] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.674350 29349 raft_consensus.cc:2454] T 67686979ffb54564986fe0733f57a8d2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:10.674451 29489 raft_consensus.cc:2454] T 67686979ffb54564986fe0733f57a8d2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:10.674584 26859 leader_election.cc:302] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:10.674726 5917 raft_consensus.cc:2781] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:10.674798 5917 raft_consensus.cc:686] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:10.674911 5917 consensus_queue.cc:237] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:10.677081 26513 catalog_manager.cc:5484] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:10.695894 6012 raft_consensus.cc:482] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:10.696014 6012 raft_consensus.cc:504] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.696498 6012 leader_election.cc:288] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:10.696913 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fba57dcf4aa4bcf964556f707d8dc99" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:10.697047 26929 raft_consensus.cc:2454] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:10.697419 29150 leader_election.cc:302] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:10.698241 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fba57dcf4aa4bcf964556f707d8dc99" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:10.698382 29349 raft_consensus.cc:2454] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:10.698544 6012 raft_consensus.cc:2781] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:10.698602 6012 raft_consensus.cc:482] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:10.698640 6012 raft_consensus.cc:3037] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.699715 6012 raft_consensus.cc:504] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.700062 6012 leader_election.cc:288] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:10.700505 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fba57dcf4aa4bcf964556f707d8dc99" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:10.700608 26929 raft_consensus.cc:3037] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.700723 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fba57dcf4aa4bcf964556f707d8dc99" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:10.700829 29349 raft_consensus.cc:3037] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:10.701519 26929 raft_consensus.cc:2454] T 1fba57dcf4aa4bcf964556f707d8dc99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:10.701793 29349 raft_consensus.cc:2454] T 1fba57dcf4aa4bcf964556f707d8dc99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:10.701869 29150 leader_election.cc:302] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:10.702175 6012 raft_consensus.cc:2781] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:10.702270 6012 raft_consensus.cc:686] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:10.702387 6012 consensus_queue.cc:237] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:10.703311 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:10.704829 26513 catalog_manager.cc:5484] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:10.710217 6103 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:10.710500 6103 tablet.cc:1620] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:10.715224 6107 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:10.715492 6107 tablet.cc:1620] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:10.716953 6104 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:10.717228 6104 tablet.cc:1620] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:10.718101 26512 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=4cff803688d4485bb465e5eaddfe2411] alter complete (version 2)
I20240502 01:59:10.775537 29195 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:10.780321 29329 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:10.783931 29468 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:10.787350 26640 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:10.787716 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:10.791062 27043 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:10.806499 5780 raft_consensus.cc:979] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: : Instructing follower 998b57e2badc4d779c586c6579149114 to start an election
I20240502 01:59:10.806586 5780 raft_consensus.cc:1067] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Signalling peer 998b57e2badc4d779c586c6579149114 to start an election
I20240502 01:59:10.807443 26661 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2"
dest_uuid: "998b57e2badc4d779c586c6579149114"
from {username='slave'} at 127.19.43.200:59383
I20240502 01:59:10.807564 26661 raft_consensus.cc:482] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:10.807612 26661 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:10.808619 26661 raft_consensus.cc:504] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:10.809310 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:10.809424 29489 raft_consensus.cc:3032] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:10.809459 29489 raft_consensus.cc:729] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:10.809602 29489 consensus_queue.cc:260] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:10.809787 29489 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:10.810711 29489 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 2.
I20240502 01:59:10.818212 27063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:10.818341 27063 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:10.818648 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8209cefcc49042b18fdd97d845d167e2" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:10.818756 29350 raft_consensus.cc:3037] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:10.819272 27063 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 2.
I20240502 01:59:10.819572 26595 leader_election.cc:302] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:10.819625 29350 raft_consensus.cc:2454] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 2.
I20240502 01:59:10.819744 6034 raft_consensus.cc:2781] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:10.819810 6034 raft_consensus.cc:686] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 2 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:10.819933 6034 consensus_queue.cc:237] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:10.820469 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:10.823688 26513 catalog_manager.cc:5484] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 2 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:10.826296 6103 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:10.826463 6107 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:10.829706 6104 tablet.cc:1722] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:10.837595 26661 leader_election.cc:288] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 2 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:10.837747 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=4cff803688d4485bb465e5eaddfe2411] alter complete (version 3)
I20240502 01:59:10.840760 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:10.842332 26908 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:10.944170 26514 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:10.944293 26514 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:10.950733 26514 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 8cb0fda261894d3c8372d5e3c15164eb
I20240502 01:59:10.951413 26640 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cb0fda261894d3c8372d5e3c15164eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:10 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:10.951424 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cb0fda261894d3c8372d5e3c15164eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:10 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:10.951534 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cb0fda261894d3c8372d5e3c15164eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:10 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:10.951691 6225 tablet_replica.cc:330] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:10.951848 6225 raft_consensus.cc:2229] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:10.952018 6226 tablet_replica.cc:330] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:10.952056 6225 raft_consensus.cc:2258] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.952430 6226 raft_consensus.cc:2229] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:10.952615 6226 raft_consensus.cc:2258] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.952754 6225 ts_tablet_manager.cc:1875] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:10.954306 6226 ts_tablet_manager.cc:1875] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:10.956198 6225 ts_tablet_manager.cc:1888] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:10.956365 6225 log.cc:1196] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/8cb0fda261894d3c8372d5e3c15164eb
I20240502 01:59:10.956705 6224 tablet_replica.cc:330] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:10.956961 6224 raft_consensus.cc:2229] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:10.957098 6225 ts_tablet_manager.cc:1909] T 8cb0fda261894d3c8372d5e3c15164eb P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:59:10.957142 6224 raft_consensus.cc:2258] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:10.957670 6226 ts_tablet_manager.cc:1888] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:10.957774 6226 log.cc:1196] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/8cb0fda261894d3c8372d5e3c15164eb
I20240502 01:59:10.957746 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table_new_name [id=4cff803688d4485bb465e5eaddfe2411]) successfully deleted
I20240502 01:59:10.957896 6224 ts_tablet_manager.cc:1875] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:10.958132 6226 ts_tablet_manager.cc:1909] T 8cb0fda261894d3c8372d5e3c15164eb P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:10.958751 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table_new_name [id=4cff803688d4485bb465e5eaddfe2411]) successfully deleted
I20240502 01:59:10.967738 6224 ts_tablet_manager.cc:1888] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:10.967868 6224 log.cc:1196] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/8cb0fda261894d3c8372d5e3c15164eb
I20240502 01:59:10.968204 6224 ts_tablet_manager.cc:1909] T 8cb0fda261894d3c8372d5e3c15164eb P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:10.968856 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 8cb0fda261894d3c8372d5e3c15164eb (table rebalancer_dynamic_table_new_name [id=4cff803688d4485bb465e5eaddfe2411]) successfully deleted
I20240502 01:59:11.029208 29329 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:11.029471 29195 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 01:59:11.035537 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:11.038017 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:11.039490 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.051223 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:11.052256 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:11.056149 5930 consensus_queue.cc:1035] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.059585 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 2399318758c4412bbe2720b051bfa85d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607]), partition=
I20240502 01:59:11.059800 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 2399318758c4412bbe2720b051bfa85d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607]), partition=
I20240502 01:59:11.059854 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2399318758c4412bbe2720b051bfa85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.060011 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2399318758c4412bbe2720b051bfa85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.062419 6240 tablet_bootstrap.cc:492] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:11.062489 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 2399318758c4412bbe2720b051bfa85d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607]), partition=
I20240502 01:59:11.062705 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2399318758c4412bbe2720b051bfa85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.063531 6240 tablet_bootstrap.cc:654] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.064626 6096 tablet_bootstrap.cc:492] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:11.065498 6096 tablet_bootstrap.cc:654] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.067420 26640 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:11.068956 27043 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:11.074126 6239 tablet_bootstrap.cc:492] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:11.075899 6240 tablet_bootstrap.cc:492] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:11.075974 6240 ts_tablet_manager.cc:1367] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:11.076390 6240 raft_consensus.cc:348] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.076486 6240 raft_consensus.cc:374] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.076512 6240 raft_consensus.cc:729] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.076613 6240 consensus_queue.cc:260] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.076792 6239 tablet_bootstrap.cc:654] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.076865 6240 ts_tablet_manager.cc:1398] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.080183 6239 tablet_bootstrap.cc:492] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:11.080318 6239 ts_tablet_manager.cc:1367] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 01:59:11.081236 5151 consensus_queue.cc:1035] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.081859 6239 raft_consensus.cc:348] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.082186 6096 tablet_bootstrap.cc:492] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:11.082273 6096 ts_tablet_manager.cc:1367] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 01:59:11.082402 6239 raft_consensus.cc:374] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.082489 6239 raft_consensus.cc:729] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.082664 6239 consensus_queue.cc:260] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.082724 6096 raft_consensus.cc:348] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.082820 6096 raft_consensus.cc:374] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.082857 6096 raft_consensus.cc:729] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.082974 6096 consensus_queue.cc:260] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.083269 6096 ts_tablet_manager.cc:1398] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.083576 6239 ts_tablet_manager.cc:1398] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:11.089560 6012 consensus_queue.cc:1035] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.099886 6012 consensus_queue.cc:1035] T 1fba57dcf4aa4bcf964556f707d8dc99 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.115859 6223 raft_consensus.cc:482] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:11.115947 6223 raft_consensus.cc:504] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.116297 6223 leader_election.cc:288] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:11.116591 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2399318758c4412bbe2720b051bfa85d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:11.116617 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2399318758c4412bbe2720b051bfa85d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:11.116711 26661 raft_consensus.cc:2454] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:11.116721 29489 raft_consensus.cc:2454] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:11.116945 26998 leader_election.cc:302] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:11.117094 6223 raft_consensus.cc:2781] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:11.117133 6223 raft_consensus.cc:482] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:11.117172 6223 raft_consensus.cc:3037] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.118170 6223 raft_consensus.cc:504] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.118475 6223 leader_election.cc:288] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:11.118844 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2399318758c4412bbe2720b051bfa85d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:11.118942 26661 raft_consensus.cc:3037] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.119319 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2399318758c4412bbe2720b051bfa85d" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:11.119412 29489 raft_consensus.cc:3037] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.119905 26661 raft_consensus.cc:2454] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:11.120183 29489 raft_consensus.cc:2454] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:11.120189 26997 leader_election.cc:302] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:11.120652 6223 raft_consensus.cc:2781] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:11.120743 6223 raft_consensus.cc:686] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:11.120857 6223 consensus_queue.cc:237] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:11.122997 26511 catalog_manager.cc:5484] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:11.132968 29350 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92"
dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:51732
I20240502 01:59:11.133133 29350 raft_consensus.cc:595] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Received request to transfer leadership
I20240502 01:59:11.196568 5151 raft_consensus.cc:979] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: : Instructing follower 0d83fdd1e6e04cf5a5602994d1dbc0df to start an election
I20240502 01:59:11.196682 5151 raft_consensus.cc:1067] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Signalling peer 0d83fdd1e6e04cf5a5602994d1dbc0df to start an election
I20240502 01:59:11.197198 29215 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92"
dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
from {username='slave'} at 127.19.43.199:33805
I20240502 01:59:11.197343 29215 raft_consensus.cc:482] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:11.197396 29215 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.197814 6158 consensus_queue.cc:1035] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.198551 29215 raft_consensus.cc:504] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.199075 29215 leader_election.cc:288] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:11.200420 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:11.200549 29349 raft_consensus.cc:3032] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:11.200592 29349 raft_consensus.cc:729] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:11.200758 29349 consensus_queue.cc:260] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.200932 29349 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.200950 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:11.201056 29487 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.201934 29349 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:11.202054 29487 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:11.202517 5291 consensus_queue.cc:1035] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.204756 29150 leader_election.cc:302] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:11.204919 6012 raft_consensus.cc:2781] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:11.204988 6012 raft_consensus.cc:686] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:11.205106 6012 consensus_queue.cc:237] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.206547 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "273b6d65e3df42ffb072e2129e832a92" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:11.206669 26794 raft_consensus.cc:3037] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.207710 26794 raft_consensus.cc:2454] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:11.208331 26510 catalog_manager.cc:5484] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 1 to 2, leader changed from 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:11.209739 29487 raft_consensus.cc:1226] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Rejecting Update request from peer 468dd27456c64d67b22e8a58af93d93d for earlier term 1. Current term is 2. Ops: []
I20240502 01:59:11.231063 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:11.234251 29487 raft_consensus.cc:1261] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:11.234356 26661 raft_consensus.cc:1261] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:11.234606 6223 consensus_queue.cc:1035] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.234735 4345 consensus_queue.cc:1035] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.236752 26511 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:11.237010 6107 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:11.237300 6107 tablet.cc:1620] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:11.237764 26513 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet 2399318758c4412bbe2720b051bfa85d (table rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607]). Expected version 1 got 0
I20240502 01:59:11.238857 26510 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 2399318758c4412bbe2720b051bfa85d (table rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607]). Expected version 1 got 0
I20240502 01:59:11.240713 6299 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:11.240993 6299 tablet.cc:1620] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:11.241323 26506 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet 2399318758c4412bbe2720b051bfa85d (table rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607]). Expected version 1 got 0
I20240502 01:59:11.243026 29487 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:11.243871 6034 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:11.251263 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 001a24f2a36141a5bde72a6f7cd56eb4 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.251314 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 50d9bfb270fb4e02bf0bcbdc48b62bc5 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.251510 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001a24f2a36141a5bde72a6f7cd56eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.251960 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 023850cc74ee44cd95e5310221503b03 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.252079 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 001a24f2a36141a5bde72a6f7cd56eb4 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.252193 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023850cc74ee44cd95e5310221503b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.252156 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 023850cc74ee44cd95e5310221503b03 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.252315 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001a24f2a36141a5bde72a6f7cd56eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.252378 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023850cc74ee44cd95e5310221503b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.253845 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 50d9bfb270fb4e02bf0bcbdc48b62bc5 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.254213 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d9bfb270fb4e02bf0bcbdc48b62bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.254354 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d9bfb270fb4e02bf0bcbdc48b62bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.254503 6240 tablet_bootstrap.cc:492] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:11.255281 6302 tablet_bootstrap.cc:492] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:11.255407 6240 tablet_bootstrap.cc:654] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.256778 6301 tablet_bootstrap.cc:492] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:11.256932 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 50d9bfb270fb4e02bf0bcbdc48b62bc5 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.257189 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d9bfb270fb4e02bf0bcbdc48b62bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.257725 6301 tablet_bootstrap.cc:654] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.256932 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 023850cc74ee44cd95e5310221503b03 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.258057 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023850cc74ee44cd95e5310221503b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.259071 6301 tablet_bootstrap.cc:492] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:11.259155 6301 ts_tablet_manager.cc:1367] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:11.259503 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 001a24f2a36141a5bde72a6f7cd56eb4 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=f40e628a2f7c449bae1be98086c51367]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:11.259565 6301 raft_consensus.cc:348] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.259658 6301 raft_consensus.cc:374] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.259685 6301 raft_consensus.cc:729] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.259758 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001a24f2a36141a5bde72a6f7cd56eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.259784 6301 consensus_queue.cc:260] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.260035 6301 ts_tablet_manager.cc:1398] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.260181 6301 tablet_bootstrap.cc:492] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:11.260536 6302 tablet_bootstrap.cc:654] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.260972 6301 tablet_bootstrap.cc:654] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.261215 6303 tablet_bootstrap.cc:492] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:11.262267 6303 tablet_bootstrap.cc:654] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.262939 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607] alter complete (version 1)
I20240502 01:59:11.263574 6301 tablet_bootstrap.cc:492] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:11.263656 6301 ts_tablet_manager.cc:1367] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:11.264078 6301 raft_consensus.cc:348] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.264190 6301 raft_consensus.cc:374] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.264220 6301 raft_consensus.cc:729] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.264326 6301 consensus_queue.cc:260] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.264585 6301 ts_tablet_manager.cc:1398] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.267807 6302 tablet_bootstrap.cc:492] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:11.267899 6302 ts_tablet_manager.cc:1367] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 01:59:11.268414 6302 raft_consensus.cc:348] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.268570 6302 raft_consensus.cc:374] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.268649 6302 raft_consensus.cc:729] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.268748 27063 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:11.268810 6302 consensus_queue.cc:260] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.269150 6302 ts_tablet_manager.cc:1398] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.269658 6034 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:11.269909 6303 tablet_bootstrap.cc:492] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:11.271701 6303 ts_tablet_manager.cc:1367] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:11.271946 29349 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:11.272257 6303 raft_consensus.cc:348] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.272356 6034 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:11.272411 6303 raft_consensus.cc:374] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.272454 6303 raft_consensus.cc:729] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.272559 6303 consensus_queue.cc:260] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.273276 6104 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:11.273553 6104 tablet.cc:1620] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:11.278237 6306 tablet_bootstrap.cc:492] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:11.280215 6309 tablet_bootstrap.cc:492] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:11.282091 6309 tablet_bootstrap.cc:654] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.282416 6303 ts_tablet_manager.cc:1398] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:59:11.282511 6240 tablet_bootstrap.cc:492] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:11.282588 6240 ts_tablet_manager.cc:1367] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240502 01:59:11.282999 6240 raft_consensus.cc:348] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.283106 6240 raft_consensus.cc:374] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.283138 6240 raft_consensus.cc:729] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.283268 6240 consensus_queue.cc:260] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.283547 6240 ts_tablet_manager.cc:1398] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.283710 6240 tablet_bootstrap.cc:492] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:11.284588 6240 tablet_bootstrap.cc:654] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.285331 6306 tablet_bootstrap.cc:654] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.288558 6309 tablet_bootstrap.cc:492] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:11.288694 6309 ts_tablet_manager.cc:1367] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 01:59:11.289304 6309 raft_consensus.cc:348] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.289462 6309 raft_consensus.cc:374] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.289537 6309 raft_consensus.cc:729] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.289744 6309 consensus_queue.cc:260] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.290840 6309 ts_tablet_manager.cc:1398] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:11.298581 6240 tablet_bootstrap.cc:492] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:11.299064 6240 ts_tablet_manager.cc:1367] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:59:11.299798 6240 raft_consensus.cc:348] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.300935 6240 raft_consensus.cc:374] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.301033 6240 raft_consensus.cc:729] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.301215 6240 consensus_queue.cc:260] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.301990 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.302690 29195 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:11.302856 6240 ts_tablet_manager.cc:1398] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:11.310598 6306 tablet_bootstrap.cc:492] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:11.310684 6306 ts_tablet_manager.cc:1367] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 01:59:11.311067 6306 raft_consensus.cc:348] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.311167 6306 raft_consensus.cc:374] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.311196 6306 raft_consensus.cc:729] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.311367 6306 consensus_queue.cc:260] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.311650 6306 ts_tablet_manager.cc:1398] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.311812 6306 tablet_bootstrap.cc:492] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:11.312553 6306 tablet_bootstrap.cc:654] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.313426 6306 tablet_bootstrap.cc:492] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:11.313503 6306 ts_tablet_manager.cc:1367] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:11.313839 6306 raft_consensus.cc:348] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.313918 6306 raft_consensus.cc:374] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.313946 6306 raft_consensus.cc:729] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.314097 6306 consensus_queue.cc:260] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.314404 6306 ts_tablet_manager.cc:1398] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.322854 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:11.324478 27043 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:11.335582 5738 raft_consensus.cc:482] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:11.335666 5738 raft_consensus.cc:504] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.335990 5738 leader_election.cc:288] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:11.336319 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023850cc74ee44cd95e5310221503b03" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:11.336429 26794 raft_consensus.cc:2454] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:11.336747 29015 leader_election.cc:302] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:11.336943 5738 raft_consensus.cc:2781] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:11.337004 5738 raft_consensus.cc:482] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:11.337038 5738 raft_consensus.cc:3037] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.337167 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023850cc74ee44cd95e5310221503b03" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:11.337282 26659 raft_consensus.cc:2454] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:11.338059 5738 raft_consensus.cc:504] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.338387 5738 leader_election.cc:288] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:11.339082 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023850cc74ee44cd95e5310221503b03" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:11.339193 26794 raft_consensus.cc:3037] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.340179 26794 raft_consensus.cc:2454] T 023850cc74ee44cd95e5310221503b03 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:11.340580 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023850cc74ee44cd95e5310221503b03" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:11.340703 26659 raft_consensus.cc:3037] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.340715 29015 leader_election.cc:302] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:11.341671 26659 raft_consensus.cc:2454] T 023850cc74ee44cd95e5310221503b03 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:11.342120 5443 raft_consensus.cc:2781] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:11.342280 5443 raft_consensus.cc:686] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:11.342411 5443 consensus_queue.cc:237] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:11.345921 6034 raft_consensus.cc:482] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:11.346004 6034 raft_consensus.cc:504] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.346369 6034 leader_election.cc:288] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:11.346737 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d9bfb270fb4e02bf0bcbdc48b62bc5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:11.346838 29080 raft_consensus.cc:2454] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:11.347455 26592 leader_election.cc:302] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:11.347743 6034 raft_consensus.cc:2781] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:11.347792 6034 raft_consensus.cc:482] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:11.347827 6034 raft_consensus.cc:3037] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.347891 26510 catalog_manager.cc:5484] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:11.348466 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d9bfb270fb4e02bf0bcbdc48b62bc5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:11.348585 26929 raft_consensus.cc:2454] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:11.348798 6034 raft_consensus.cc:504] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.349097 6034 leader_election.cc:288] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:11.349428 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d9bfb270fb4e02bf0bcbdc48b62bc5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:11.349535 26929 raft_consensus.cc:3037] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.350073 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d9bfb270fb4e02bf0bcbdc48b62bc5" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:11.350170 29080 raft_consensus.cc:3037] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.350484 26929 raft_consensus.cc:2454] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:11.350754 26595 leader_election.cc:302] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:11.350885 6034 raft_consensus.cc:2781] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:11.350940 6034 raft_consensus.cc:686] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:11.351032 6034 consensus_queue.cc:237] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:11.351064 29080 raft_consensus.cc:2454] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:11.353250 26513 catalog_manager.cc:5484] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:11.354133 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:11.354566 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:11.356148 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:11.374980 26513 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:11.380509 6107 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:11.380797 6107 tablet.cc:1620] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:11.381068 6104 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:11.381345 6104 tablet.cc:1620] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:11.386808 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.388988 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=2b23cd5bcf42474082f97db9ff61d607] alter complete (version 2)
I20240502 01:59:11.389716 6299 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:11.390002 6299 tablet.cc:1620] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:11.501310 26506 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:11.512526 6107 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:11.513015 6104 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:11.519184 6299 tablet.cc:1722] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:11.523097 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=2b23cd5bcf42474082f97db9ff61d607] alter complete (version 3)
I20240502 01:59:11.547456 5818 raft_consensus.cc:482] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:11.547555 5818 raft_consensus.cc:504] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.547900 5818 leader_election.cc:288] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:11.548202 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a24f2a36141a5bde72a6f7cd56eb4" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:11.548198 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a24f2a36141a5bde72a6f7cd56eb4" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:11.548341 26929 raft_consensus.cc:2454] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:11.548345 29350 raft_consensus.cc:2454] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:11.548602 29150 leader_election.cc:302] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:11.548820 5818 raft_consensus.cc:2781] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:11.548871 5818 raft_consensus.cc:482] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:11.548907 5818 raft_consensus.cc:3037] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.549973 5818 raft_consensus.cc:504] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.550313 5818 leader_election.cc:288] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:11.550583 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a24f2a36141a5bde72a6f7cd56eb4" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:11.550616 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001a24f2a36141a5bde72a6f7cd56eb4" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:11.550686 29350 raft_consensus.cc:3037] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.550722 26929 raft_consensus.cc:3037] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.551589 29350 raft_consensus.cc:2454] T 001a24f2a36141a5bde72a6f7cd56eb4 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:11.551610 26929 raft_consensus.cc:2454] T 001a24f2a36141a5bde72a6f7cd56eb4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:11.551853 29147 leader_election.cc:302] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:59:11.552035 5818 raft_consensus.cc:2781] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:11.552124 5818 raft_consensus.cc:686] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:11.552249 5818 consensus_queue.cc:237] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:11.554384 26506 catalog_manager.cc:5484] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:11.568634 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.582645 29195 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:11.587492 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:11.588439 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:11.594468 27043 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:11.595047 29059 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 01:59:11.614166 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:11.626081 26506 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:11.626224 26506 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:11.628424 26506 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 2399318758c4412bbe2720b051bfa85d
I20240502 01:59:11.629065 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet 2399318758c4412bbe2720b051bfa85d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:11 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:11.629065 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet 2399318758c4412bbe2720b051bfa85d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:11 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:11.629099 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 2399318758c4412bbe2720b051bfa85d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:11 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:11.631063 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.631537 6388 tablet_replica.cc:330] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:11.637099 6388 raft_consensus.cc:2229] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:11.637281 6388 raft_consensus.cc:2258] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:11.640611 6388 ts_tablet_manager.cc:1875] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:11.642889 6388 ts_tablet_manager.cc:1888] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:11.647001 6387 tablet_replica.cc:330] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:11.647243 6389 tablet_replica.cc:330] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:11.647434 6387 raft_consensus.cc:2229] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:11.647600 6389 raft_consensus.cc:2229] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:11.648240 6387 raft_consensus.cc:2258] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:11.647818 6389 raft_consensus.cc:2258] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:11.650287 6389 ts_tablet_manager.cc:1875] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:11.650487 6387 ts_tablet_manager.cc:1875] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:11.650801 6388 log.cc:1196] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/2399318758c4412bbe2720b051bfa85d
I20240502 01:59:11.653275 6388 ts_tablet_manager.cc:1909] T 2399318758c4412bbe2720b051bfa85d P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:59:11.653968 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 2399318758c4412bbe2720b051bfa85d (table rebalancer_dynamic_table_new_name [id=2b23cd5bcf42474082f97db9ff61d607]) successfully deleted
I20240502 01:59:11.656455 6387 ts_tablet_manager.cc:1888] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:11.656517 6387 log.cc:1196] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/2399318758c4412bbe2720b051bfa85d
I20240502 01:59:11.656769 6387 ts_tablet_manager.cc:1909] T 2399318758c4412bbe2720b051bfa85d P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:11.657343 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet 2399318758c4412bbe2720b051bfa85d (table rebalancer_dynamic_table_new_name [id=2b23cd5bcf42474082f97db9ff61d607]) successfully deleted
I20240502 01:59:11.658960 6389 ts_tablet_manager.cc:1888] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:11.659034 6389 log.cc:1196] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/2399318758c4412bbe2720b051bfa85d
I20240502 01:59:11.659328 6389 ts_tablet_manager.cc:1909] T 2399318758c4412bbe2720b051bfa85d P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:11.660084 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 2399318758c4412bbe2720b051bfa85d (table rebalancer_dynamic_table_new_name [id=2b23cd5bcf42474082f97db9ff61d607]) successfully deleted
I20240502 01:59:11.670758 29080 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29"
dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56468
I20240502 01:59:11.670886 29080 raft_consensus.cc:595] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Received request to transfer leadership
I20240502 01:59:11.695490 26793 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:11.695870 6012 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:11.732904 26512 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:11.739650 26775 tablet_service.cc:1467] Processing CreateTablet for tablet e1d82925762b42f7af31575d4ccf18f3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2]), partition=
I20240502 01:59:11.739921 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d82925762b42f7af31575d4ccf18f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.740147 29468 tablet_service.cc:1467] Processing CreateTablet for tablet e1d82925762b42f7af31575d4ccf18f3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2]), partition=
I20240502 01:59:11.740379 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d82925762b42f7af31575d4ccf18f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.741815 6303 tablet_bootstrap.cc:492] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:11.742107 27043 tablet_service.cc:1467] Processing CreateTablet for tablet e1d82925762b42f7af31575d4ccf18f3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2]), partition=
I20240502 01:59:11.742343 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d82925762b42f7af31575d4ccf18f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:11.742910 6303 tablet_bootstrap.cc:654] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.743919 6303 tablet_bootstrap.cc:492] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:11.744006 6303 ts_tablet_manager.cc:1367] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:11.744419 6303 raft_consensus.cc:348] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.744514 6303 raft_consensus.cc:374] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.744544 6303 raft_consensus.cc:729] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.744648 6303 consensus_queue.cc:260] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.744917 6303 ts_tablet_manager.cc:1398] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.747803 6434 tablet_bootstrap.cc:492] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:11.748720 6434 tablet_bootstrap.cc:654] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.750018 6434 tablet_bootstrap.cc:492] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:11.750142 6434 ts_tablet_manager.cc:1367] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:11.750559 6434 raft_consensus.cc:348] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.750656 6434 raft_consensus.cc:374] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.750687 6434 raft_consensus.cc:729] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.750797 6434 consensus_queue.cc:260] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.751055 6434 ts_tablet_manager.cc:1398] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:11.752173 6432 tablet_bootstrap.cc:492] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:11.752934 6432 tablet_bootstrap.cc:654] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:11.762010 6432 tablet_bootstrap.cc:492] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:11.762146 6432 ts_tablet_manager.cc:1367] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240502 01:59:11.762579 6432 raft_consensus.cc:348] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.762670 6432 raft_consensus.cc:374] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:11.762697 6432 raft_consensus.cc:729] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:11.762832 6432 consensus_queue.cc:260] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.763101 6432 ts_tablet_manager.cc:1398] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:11.766680 6328 raft_consensus.cc:979] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: : Instructing follower 68e5b1eeb04448fea818a14991eab87c to start an election
I20240502 01:59:11.766762 6328 raft_consensus.cc:1067] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Signalling peer 68e5b1eeb04448fea818a14991eab87c to start an election
I20240502 01:59:11.767226 27062 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29"
dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
from {username='slave'} at 127.19.43.197:55881
I20240502 01:59:11.767354 27062 raft_consensus.cc:482] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:11.767397 27062 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.768169 27062 raft_consensus.cc:504] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:11.768534 27062 leader_election.cc:288] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:11.768852 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:11.768961 29215 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.768990 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:11.768980 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8afd4c5362464c4cba43d5ef28b50c29" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:11.769088 29349 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.769088 29079 raft_consensus.cc:3032] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:11.769124 29079 raft_consensus.cc:729] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:11.769249 29079 consensus_queue.cc:260] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:11.769457 29079 raft_consensus.cc:3037] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:11.769976 29215 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 2.
I20240502 01:59:11.770116 29349 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 2.
I20240502 01:59:11.770476 29079 raft_consensus.cc:2454] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 2.
I20240502 01:59:11.770681 26997 leader_election.cc:302] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:11.770920 6223 raft_consensus.cc:2781] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:11.771011 6223 raft_consensus.cc:686] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 2 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:11.771132 6223 consensus_queue.cc:237] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:11.773473 26513 catalog_manager.cc:5484] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 1 to 2, leader changed from f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 2 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:11.781379 6386 consensus_queue.cc:1035] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.787789 4649 consensus_queue.cc:1035] T 50d9bfb270fb4e02bf0bcbdc48b62bc5 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.802649 29487 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:11.803421 5818 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:11.809267 29195 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:11.815263 29349 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 01:59:11.817430 6456 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:11.824729 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:11.828884 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.830982 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:11.836125 4345 raft_consensus.cc:482] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:11.836220 4345 raft_consensus.cc:504] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.836614 4345 leader_election.cc:288] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:11.836925 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d82925762b42f7af31575d4ccf18f3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:11.837042 29487 raft_consensus.cc:2454] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:11.837034 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d82925762b42f7af31575d4ccf18f3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:11.837159 26793 raft_consensus.cc:2454] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:11.837435 26998 leader_election.cc:302] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:11.837829 6223 raft_consensus.cc:2781] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:11.837879 6223 raft_consensus.cc:482] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:11.837909 6223 raft_consensus.cc:3037] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.838908 6223 raft_consensus.cc:504] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.839244 6223 leader_election.cc:288] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:11.840054 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d82925762b42f7af31575d4ccf18f3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:11.840159 26793 raft_consensus.cc:3037] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.840307 29487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d82925762b42f7af31575d4ccf18f3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:11.840406 29487 raft_consensus.cc:3037] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:11.841077 26793 raft_consensus.cc:2454] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:11.841286 29487 raft_consensus.cc:2454] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:11.841374 26997 leader_election.cc:302] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:11.841864 4345 raft_consensus.cc:2781] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:11.841960 4345 raft_consensus.cc:686] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:11.842095 4345 consensus_queue.cc:237] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:11.842375 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:11.843544 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:11.844058 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:11.844760 26511 catalog_manager.cc:5484] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:11.859886 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:11.878829 5443 consensus_queue.cc:1035] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.888187 5443 consensus_queue.cc:1035] T 023850cc74ee44cd95e5310221503b03 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.958065 6012 consensus_queue.cc:1035] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.963186 5818 consensus_queue.cc:1035] T 001a24f2a36141a5bde72a6f7cd56eb4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.968753 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:11.973899 26793 raft_consensus.cc:1261] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:11.974458 4345 consensus_queue.cc:1035] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.974642 29488 raft_consensus.cc:1261] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:11.975003 4345 consensus_queue.cc:1035] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:11.979266 26513 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet e1d82925762b42f7af31575d4ccf18f3 (table rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2]). Expected version 1 got 0
I20240502 01:59:11.980033 26511 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet e1d82925762b42f7af31575d4ccf18f3 (table rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2]). Expected version 1 got 0
I20240502 01:59:11.981766 6107 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:11.982076 6107 tablet.cc:1620] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:11.989993 6484 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:11.990301 6484 tablet.cc:1620] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:11.991480 26513 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet e1d82925762b42f7af31575d4ccf18f3 (table rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2]). Expected version 1 got 0
I20240502 01:59:11.991977 6299 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:11.992259 6299 tablet.cc:1620] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:11.992254 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2] alter complete (version 1)
I20240502 01:59:12.081696 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:12.089402 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 79e7d37ead3c4198bc2a9ae7eb28ebcf (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.089412 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 79e7d37ead3c4198bc2a9ae7eb28ebcf (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.089707 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e7d37ead3c4198bc2a9ae7eb28ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.089713 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e7d37ead3c4198bc2a9ae7eb28ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.089942 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 7d48607cfc6c4757ab2df41edb6da47d (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.090212 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d48607cfc6c4757ab2df41edb6da47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.090297 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 4734862e88bb47eb8a3b76f0310f3de7 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.090544 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4734862e88bb47eb8a3b76f0310f3de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.092187 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 4734862e88bb47eb8a3b76f0310f3de7 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.092272 6511 tablet_bootstrap.cc:492] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:12.092407 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4734862e88bb47eb8a3b76f0310f3de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.093230 6511 tablet_bootstrap.cc:654] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.093324 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 7d48607cfc6c4757ab2df41edb6da47d (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.093578 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d48607cfc6c4757ab2df41edb6da47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.093837 6512 tablet_bootstrap.cc:492] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:12.094969 6512 tablet_bootstrap.cc:654] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.095129 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 7d48607cfc6c4757ab2df41edb6da47d (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.095343 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d48607cfc6c4757ab2df41edb6da47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.095855 6432 tablet_bootstrap.cc:492] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:12.096191 6512 tablet_bootstrap.cc:492] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:12.096293 6512 ts_tablet_manager.cc:1367] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:12.096801 6432 tablet_bootstrap.cc:654] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.096758 6512 raft_consensus.cc:348] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.096854 6512 raft_consensus.cc:374] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.096887 6512 raft_consensus.cc:729] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.096995 6512 consensus_queue.cc:260] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.097261 6512 ts_tablet_manager.cc:1398] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:12.097421 6512 tablet_bootstrap.cc:492] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:12.098275 6512 tablet_bootstrap.cc:654] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.098806 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 4734862e88bb47eb8a3b76f0310f3de7 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.099071 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4734862e88bb47eb8a3b76f0310f3de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.099217 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:12.099261 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 79e7d37ead3c4198bc2a9ae7eb28ebcf (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=5a0dca4eeee049ce95fe11731653435c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:12.099500 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e7d37ead3c4198bc2a9ae7eb28ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.101117 6432 tablet_bootstrap.cc:492] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:12.101193 6432 ts_tablet_manager.cc:1367] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 01:59:12.101261 6434 tablet_bootstrap.cc:492] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:12.101639 6432 raft_consensus.cc:348] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.101748 6432 raft_consensus.cc:374] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.101778 6432 raft_consensus.cc:729] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.101900 6432 consensus_queue.cc:260] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.102195 6434 tablet_bootstrap.cc:654] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.102191 6432 ts_tablet_manager.cc:1398] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.102880 6512 tablet_bootstrap.cc:492] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:12.102945 6512 ts_tablet_manager.cc:1367] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:12.103332 6512 raft_consensus.cc:348] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.103420 6512 raft_consensus.cc:374] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.103444 6512 raft_consensus.cc:729] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.103539 6512 consensus_queue.cc:260] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.103916 6512 ts_tablet_manager.cc:1398] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.108683 6107 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:12.108933 6107 tablet.cc:1620] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:12.110195 6513 tablet_bootstrap.cc:492] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:12.110343 6299 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:12.110422 6514 tablet_bootstrap.cc:492] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:12.110599 6299 tablet.cc:1620] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:12.112151 6514 tablet_bootstrap.cc:654] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.112877 6513 tablet_bootstrap.cc:654] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.114655 6434 tablet_bootstrap.cc:492] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:12.114735 6434 ts_tablet_manager.cc:1367] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:12.115144 6434 raft_consensus.cc:348] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.115239 6434 raft_consensus.cc:374] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.115269 6434 raft_consensus.cc:729] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.115413 6434 consensus_queue.cc:260] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.115715 6434 ts_tablet_manager.cc:1398] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:12.115878 6434 tablet_bootstrap.cc:492] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:12.116225 6484 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:12.116523 6484 tablet.cc:1620] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:12.116641 6434 tablet_bootstrap.cc:654] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.117882 6511 tablet_bootstrap.cc:492] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:12.117938 6434 tablet_bootstrap.cc:492] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:12.117965 6511 ts_tablet_manager.cc:1367] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 01:59:12.118003 6434 ts_tablet_manager.cc:1367] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:12.118403 6434 raft_consensus.cc:348] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.118424 6511 raft_consensus.cc:348] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.118494 6434 raft_consensus.cc:374] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.118525 6434 raft_consensus.cc:729] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.118525 6511 raft_consensus.cc:374] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.118556 6511 raft_consensus.cc:729] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.118618 6434 consensus_queue.cc:260] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.118685 6511 consensus_queue.cc:260] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.118871 6434 ts_tablet_manager.cc:1398] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.119000 6511 ts_tablet_manager.cc:1398] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.119167 6511 tablet_bootstrap.cc:492] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:12.119516 6513 tablet_bootstrap.cc:492] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:12.119590 6513 ts_tablet_manager.cc:1367] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:59:12.120020 6513 raft_consensus.cc:348] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.120127 6511 tablet_bootstrap.cc:654] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.120132 6513 raft_consensus.cc:374] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.120167 6513 raft_consensus.cc:729] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.120281 6513 consensus_queue.cc:260] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.120556 6513 ts_tablet_manager.cc:1398] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:12.122464 6514 tablet_bootstrap.cc:492] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:12.122777 6514 ts_tablet_manager.cc:1367] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:59:12.122989 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=67c5d6ad1f77431595aba74b5e9a93a2] alter complete (version 2)
I20240502 01:59:12.123350 6514 raft_consensus.cc:348] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.123523 6514 raft_consensus.cc:374] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.123607 6514 raft_consensus.cc:729] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.123768 6514 consensus_queue.cc:260] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.124207 6514 ts_tablet_manager.cc:1398] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.135553 6511 tablet_bootstrap.cc:492] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:12.135635 6511 ts_tablet_manager.cc:1367] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:59:12.136034 6511 raft_consensus.cc:348] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.136147 6511 raft_consensus.cc:374] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.136174 6511 raft_consensus.cc:729] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.136269 6511 consensus_queue.cc:260] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.136529 6511 ts_tablet_manager.cc:1398] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.160039 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:12.172340 29196 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:12.175629 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:12.176286 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:12.205946 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:12.210225 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:12.223652 29213 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:12.225569 6223 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:12.227506 29059 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:12.229831 26509 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:12.231510 5443 raft_consensus.cc:482] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:12.231607 5443 raft_consensus.cc:504] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.231977 5443 leader_election.cc:288] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:12.232534 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d48607cfc6c4757ab2df41edb6da47d" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:12.232654 26661 raft_consensus.cc:2454] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:12.234102 29015 leader_election.cc:302] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:12.234730 5443 raft_consensus.cc:2781] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:12.234781 5443 raft_consensus.cc:482] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:12.234810 5443 raft_consensus.cc:3037] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.235803 5443 raft_consensus.cc:504] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.236142 5443 leader_election.cc:288] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:12.236723 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d48607cfc6c4757ab2df41edb6da47d" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:12.236828 29489 raft_consensus.cc:3037] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.237774 29489 raft_consensus.cc:2454] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:12.238075 29016 leader_election.cc:302] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:12.238216 5443 raft_consensus.cc:2781] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:12.238277 5443 raft_consensus.cc:686] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:12.238381 5443 consensus_queue.cc:237] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.241168 26509 catalog_manager.cc:5484] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:12.241217 6107 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:12.243568 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d48607cfc6c4757ab2df41edb6da47d" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:12.243695 26661 raft_consensus.cc:3037] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.244077 6299 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:12.244679 26661 raft_consensus.cc:2454] T 7d48607cfc6c4757ab2df41edb6da47d P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:12.247396 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d48607cfc6c4757ab2df41edb6da47d" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:12.247499 29488 raft_consensus.cc:2362] T 7d48607cfc6c4757ab2df41edb6da47d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1. Re-sending same reply.
I20240502 01:59:12.251878 6484 tablet.cc:1722] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:12.255501 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=67c5d6ad1f77431595aba74b5e9a93a2] alter complete (version 3)
I20240502 01:59:12.258387 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:12.278600 29079 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:12.279131 6373 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:12.282076 29349 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:12.282469 6373 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:12.302408 26927 consensus_queue.cc:237] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:12.303550 29488 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:12.303588 29213 raft_consensus.cc:1261] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:12.303926 5291 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:12.304119 6536 consensus_queue.cc:1035] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:12.305907 5291 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:12.306439 29488 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:12.309074 26511 catalog_manager.cc:5484] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 6 to 7, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:12.311507 29213 raft_consensus.cc:2932] T 626774246b9445bc838427358e08f2cd P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:12.313272 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 626774246b9445bc838427358e08f2cd with delete_type TABLET_DATA_TOMBSTONED (TS b290c1bf893444078ed721f25995d124 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:12.315932 6559 tablet_replica.cc:330] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:12.316154 6559 raft_consensus.cc:2229] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:12.316534 6559 raft_consensus.cc:2258] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:12.317081 6559 ts_tablet_manager.cc:1875] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:12.318965 6559 ts_tablet_manager.cc:1888] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 01:59:12.319029 6559 log.cc:1196] T 626774246b9445bc838427358e08f2cd P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/626774246b9445bc838427358e08f2cd
I20240502 01:59:12.319515 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 626774246b9445bc838427358e08f2cd (table rebalance_test_table_7 [id=e8b7140ce14c490fa251594750e1da52]) successfully deleted
I20240502 01:59:12.353746 6373 raft_consensus.cc:482] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:12.353833 6373 raft_consensus.cc:504] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.354214 6373 leader_election.cc:288] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:12.354480 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e7d37ead3c4198bc2a9ae7eb28ebcf" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:12.354535 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e7d37ead3c4198bc2a9ae7eb28ebcf" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:12.354620 26659 raft_consensus.cc:2454] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:12.354647 29213 raft_consensus.cc:2454] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:12.354905 26997 leader_election.cc:302] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:12.355070 6373 raft_consensus.cc:2781] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:12.355142 6373 raft_consensus.cc:482] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:12.355181 6373 raft_consensus.cc:3037] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.356115 6373 raft_consensus.cc:504] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.356446 6373 leader_election.cc:288] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:12.356751 29213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e7d37ead3c4198bc2a9ae7eb28ebcf" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:12.356751 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e7d37ead3c4198bc2a9ae7eb28ebcf" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:12.356858 26659 raft_consensus.cc:3037] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.356858 29213 raft_consensus.cc:3037] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.357761 29213 raft_consensus.cc:2454] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:12.357761 26659 raft_consensus.cc:2454] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:12.358137 26997 leader_election.cc:302] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:12.358337 6373 raft_consensus.cc:2781] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:12.358434 6373 raft_consensus.cc:686] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:12.358554 6373 consensus_queue.cc:237] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:12.358734 26509 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:12.358830 26509 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:12.361001 26509 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet e1d82925762b42f7af31575d4ccf18f3
I20240502 01:59:12.361630 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet e1d82925762b42f7af31575d4ccf18f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:12 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:12.361616 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet e1d82925762b42f7af31575d4ccf18f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:12 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:12.361793 6559 tablet_replica.cc:330] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:12.361680 26507 catalog_manager.cc:5484] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:12.361927 6559 raft_consensus.cc:2229] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:12.362063 6559 raft_consensus.cc:2258] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:12.362248 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet e1d82925762b42f7af31575d4ccf18f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:12 UTC) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:12.362532 6584 tablet_replica.cc:330] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:12.362624 6559 ts_tablet_manager.cc:1875] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:12.362655 6584 raft_consensus.cc:2229] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:12.362757 6584 raft_consensus.cc:2258] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:12.363561 6583 tablet_replica.cc:330] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:12.363727 6583 raft_consensus.cc:2229] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:12.363896 6584 ts_tablet_manager.cc:1875] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:12.363926 6583 raft_consensus.cc:2258] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:12.364583 6583 ts_tablet_manager.cc:1875] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:12.364684 6559 ts_tablet_manager.cc:1888] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:12.364758 6559 log.cc:1196] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/e1d82925762b42f7af31575d4ccf18f3
I20240502 01:59:12.365010 6559 ts_tablet_manager.cc:1909] T e1d82925762b42f7af31575d4ccf18f3 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:12.365737 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet e1d82925762b42f7af31575d4ccf18f3 (table rebalancer_dynamic_table_new_name [id=67c5d6ad1f77431595aba74b5e9a93a2]) successfully deleted
I20240502 01:59:12.366539 6584 ts_tablet_manager.cc:1888] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:12.366617 6584 log.cc:1196] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/e1d82925762b42f7af31575d4ccf18f3
I20240502 01:59:12.367123 6583 ts_tablet_manager.cc:1888] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:12.367169 6584 ts_tablet_manager.cc:1909] T e1d82925762b42f7af31575d4ccf18f3 P 7c693d0933c64d9babc07f404df45cd1: Deleting consensus metadata
I20240502 01:59:12.367240 6583 log.cc:1196] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/e1d82925762b42f7af31575d4ccf18f3
I20240502 01:59:12.367552 6583 ts_tablet_manager.cc:1909] T e1d82925762b42f7af31575d4ccf18f3 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:12.368001 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet e1d82925762b42f7af31575d4ccf18f3 (table rebalancer_dynamic_table_new_name [id=67c5d6ad1f77431595aba74b5e9a93a2]) successfully deleted
I20240502 01:59:12.368157 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet e1d82925762b42f7af31575d4ccf18f3 (table rebalancer_dynamic_table_new_name [id=67c5d6ad1f77431595aba74b5e9a93a2]) successfully deleted
I20240502 01:59:12.433315 5818 raft_consensus.cc:482] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:12.433414 5818 raft_consensus.cc:504] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.433717 5818 leader_election.cc:288] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:12.434067 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4734862e88bb47eb8a3b76f0310f3de7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:12.434126 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4734862e88bb47eb8a3b76f0310f3de7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:12.434216 29349 raft_consensus.cc:2454] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:12.434260 27064 raft_consensus.cc:2454] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:12.434451 29147 leader_election.cc:302] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:59:12.434650 5818 raft_consensus.cc:2781] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:12.434705 5818 raft_consensus.cc:482] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:12.434729 5818 raft_consensus.cc:3037] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.435768 5818 raft_consensus.cc:504] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.436100 5818 leader_election.cc:288] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:12.436769 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4734862e88bb47eb8a3b76f0310f3de7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:12.436880 29349 raft_consensus.cc:3037] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.437880 29349 raft_consensus.cc:2454] T 4734862e88bb47eb8a3b76f0310f3de7 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:12.438160 29147 leader_election.cc:302] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:59:12.438697 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4734862e88bb47eb8a3b76f0310f3de7" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:12.438809 27064 raft_consensus.cc:3037] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.439817 27064 raft_consensus.cc:2454] T 4734862e88bb47eb8a3b76f0310f3de7 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:12.440522 6456 raft_consensus.cc:2781] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:12.440603 6456 raft_consensus.cc:686] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:12.440733 6456 consensus_queue.cc:237] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:12.443742 26511 catalog_manager.cc:5484] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:12.463800 26506 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:12.472851 27043 tablet_service.cc:1467] Processing CreateTablet for tablet b9ca883d28e147cc812ad22aa7ac9d73 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98]), partition=
I20240502 01:59:12.473083 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ca883d28e147cc812ad22aa7ac9d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.473098 29059 tablet_service.cc:1467] Processing CreateTablet for tablet b9ca883d28e147cc812ad22aa7ac9d73 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98]), partition=
I20240502 01:59:12.473336 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ca883d28e147cc812ad22aa7ac9d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.474995 6434 tablet_bootstrap.cc:492] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:12.475713 26641 tablet_service.cc:1467] Processing CreateTablet for tablet b9ca883d28e147cc812ad22aa7ac9d73 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98]), partition=
I20240502 01:59:12.475878 6434 tablet_bootstrap.cc:654] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.475934 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ca883d28e147cc812ad22aa7ac9d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:12.477833 6434 tablet_bootstrap.cc:492] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:12.477921 6434 ts_tablet_manager.cc:1367] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:12.478399 6434 raft_consensus.cc:348] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.478513 6434 raft_consensus.cc:374] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.478547 6434 raft_consensus.cc:729] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.478648 6434 consensus_queue.cc:260] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.478873 6434 ts_tablet_manager.cc:1398] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:12.479633 6513 tablet_bootstrap.cc:492] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:12.480499 6513 tablet_bootstrap.cc:654] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.481124 6511 tablet_bootstrap.cc:492] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:12.481950 6511 tablet_bootstrap.cc:654] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:12.485706 6511 tablet_bootstrap.cc:492] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:12.485786 6511 ts_tablet_manager.cc:1367] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 01:59:12.486284 6511 raft_consensus.cc:348] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.486397 6511 raft_consensus.cc:374] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.486431 6511 raft_consensus.cc:729] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.486536 6511 consensus_queue.cc:260] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.486819 6511 ts_tablet_manager.cc:1398] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:12.490104 6513 tablet_bootstrap.cc:492] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:12.490197 6513 ts_tablet_manager.cc:1367] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:12.490631 6513 raft_consensus.cc:348] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.490749 6513 raft_consensus.cc:374] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:12.490789 6513 raft_consensus.cc:729] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:12.490900 6513 consensus_queue.cc:260] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.491176 6513 ts_tablet_manager.cc:1398] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:12.529331 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:12.531682 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:12.533826 29059 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 01:59:12.534332 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:12.534562 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:12.541167 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:12.541189 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:12.545217 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:12.641660 29349 consensus_queue.cc:237] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.642812 26659 raft_consensus.cc:1261] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:12.642858 29213 raft_consensus.cc:1261] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:12.643210 6677 consensus_queue.cc:1035] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:12.643388 6678 consensus_queue.cc:1035] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:12.646067 6677 raft_consensus.cc:2932] T 78609c45dc0c4e1b838bb737ebf7e034 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:12.646453 26659 raft_consensus.cc:2932] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:12.646613 29213 raft_consensus.cc:2932] T 78609c45dc0c4e1b838bb737ebf7e034 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:12.649646 26512 catalog_manager.cc:5484] T 78609c45dc0c4e1b838bb737ebf7e034 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 2 to 3, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:12.652535 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 78609c45dc0c4e1b838bb737ebf7e034 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:12.652750 6583 tablet_replica.cc:330] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:12.652899 6583 raft_consensus.cc:2229] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:12.653020 6583 raft_consensus.cc:2258] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:12.653455 6583 ts_tablet_manager.cc:1875] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:12.655591 6583 ts_tablet_manager.cc:1888] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:12.655673 6583 log.cc:1196] T 78609c45dc0c4e1b838bb737ebf7e034 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/78609c45dc0c4e1b838bb737ebf7e034
I20240502 01:59:12.656128 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 78609c45dc0c4e1b838bb737ebf7e034 (table rebalancer_extra_table_12 [id=d35b9b3b5c44483f9111e1bbe6f89415]) successfully deleted
I20240502 01:59:12.664484 5443 consensus_queue.cc:1035] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:12.671909 5443 consensus_queue.cc:1035] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:12.755744 4345 consensus_queue.cc:1035] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:12.761623 4345 consensus_queue.cc:1035] T 79e7d37ead3c4198bc2a9ae7eb28ebcf P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:12.860942 4345 raft_consensus.cc:482] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:12.860939 6386 raft_consensus.cc:482] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:12.861038 4345 raft_consensus.cc:504] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.861049 6386 raft_consensus.cc:504] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.861388 4345 leader_election.cc:288] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:12.861461 6386 leader_election.cc:288] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:12.861637 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:12.861723 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:12.861728 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:12.861788 29079 raft_consensus.cc:2454] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:12.861826 27062 raft_consensus.cc:2454] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:12.861840 26659 raft_consensus.cc:2454] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:12.861840 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:12.861945 29080 raft_consensus.cc:2454] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:12.862087 26995 leader_election.cc:302] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:12.862210 26592 leader_election.cc:302] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:12.862282 4345 raft_consensus.cc:2781] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:12.862349 4345 raft_consensus.cc:482] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:12.862383 4345 raft_consensus.cc:3037] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.862397 6386 raft_consensus.cc:2781] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:12.862457 6386 raft_consensus.cc:482] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:12.862500 6386 raft_consensus.cc:3037] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.863229 4345 raft_consensus.cc:504] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.863260 6386 raft_consensus.cc:504] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.863548 4345 leader_election.cc:288] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:12.863559 6386 leader_election.cc:288] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:12.863847 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:12.863873 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:12.863981 27062 raft_consensus.cc:2379] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 998b57e2badc4d779c586c6579149114 in current term 1: Already voted for candidate 68e5b1eeb04448fea818a14991eab87c in this term.
I20240502 01:59:12.863996 26659 raft_consensus.cc:2379] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68e5b1eeb04448fea818a14991eab87c in current term 1: Already voted for candidate 998b57e2badc4d779c586c6579149114 in this term.
I20240502 01:59:12.864019 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:12.864106 29080 raft_consensus.cc:3037] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:12.864147 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ca883d28e147cc812ad22aa7ac9d73" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:12.864480 26995 leader_election.cc:302] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:12.864625 4345 raft_consensus.cc:2726] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 01:59:12.865063 29080 raft_consensus.cc:2454] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:12.865325 26592 leader_election.cc:302] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters: 68e5b1eeb04448fea818a14991eab87c
I20240502 01:59:12.865525 6386 raft_consensus.cc:2781] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:12.865600 6386 raft_consensus.cc:686] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:12.865797 6386 consensus_queue.cc:237] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:12.867978 26507 catalog_manager.cc:5484] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:12.889847 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:12.894418 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:12.898447 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:12.898593 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:12.902573 26641 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:12.905145 29059 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 01:59:12.914180 29329 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:12.918635 26775 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:12.993449 5818 consensus_queue.cc:1035] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:12.998039 5818 consensus_queue.cc:1035] T 4734862e88bb47eb8a3b76f0310f3de7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:12.998344 29348 consensus_queue.cc:237] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:12.999594 26927 raft_consensus.cc:1261] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:13.000006 6677 consensus_queue.cc:1035] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:13.000245 29080 raft_consensus.cc:1261] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:13.002152 6678 raft_consensus.cc:2932] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:13.002537 26929 raft_consensus.cc:2932] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:13.004091 6679 consensus_queue.cc:1035] T 089d9dc33a594712ac04ad28a0cd37b1 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:13.004894 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:13.005195 29079 raft_consensus.cc:2932] T 089d9dc33a594712ac04ad28a0cd37b1 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:13.005820 26512 catalog_manager.cc:5484] T 089d9dc33a594712ac04ad28a0cd37b1 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 2 to 3, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:13.010500 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet 089d9dc33a594712ac04ad28a0cd37b1 with delete_type TABLET_DATA_TOMBSTONED (TS 7c693d0933c64d9babc07f404df45cd1 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:13.010803 6769 tablet_replica.cc:330] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:13.010959 6769 raft_consensus.cc:2229] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:13.011116 6769 raft_consensus.cc:2258] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:13.011571 6769 ts_tablet_manager.cc:1875] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:13.013448 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 3128a1fe3f5d49cfb334f1f1e3c6c147 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.013697 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128a1fe3f5d49cfb334f1f1e3c6c147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.013747 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 43457aa7db604c8192ea4bc1f8885c3c (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.013960 26773 tablet_service.cc:1467] Processing CreateTablet for tablet 3128a1fe3f5d49cfb334f1f1e3c6c147 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.014544 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 43457aa7db604c8192ea4bc1f8885c3c (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.014796 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43457aa7db604c8192ea4bc1f8885c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.013960 26774 tablet_service.cc:1467] Processing CreateTablet for tablet d0ffb725f2d843f8a03d8d0b711ce02d (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.013975 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43457aa7db604c8192ea4bc1f8885c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.015599 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128a1fe3f5d49cfb334f1f1e3c6c147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.016779 26908 tablet_service.cc:1467] Processing CreateTablet for tablet d0ffb725f2d843f8a03d8d0b711ce02d (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.016981 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ffb725f2d843f8a03d8d0b711ce02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.017179 6770 tablet_bootstrap.cc:492] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:13.017557 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ffb725f2d843f8a03d8d0b711ce02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.018357 29468 tablet_service.cc:1467] Processing CreateTablet for tablet d0ffb725f2d843f8a03d8d0b711ce02d (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.018590 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ffb725f2d843f8a03d8d0b711ce02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.019308 6770 tablet_bootstrap.cc:654] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.020445 29329 tablet_service.cc:1467] Processing CreateTablet for tablet 43457aa7db604c8192ea4bc1f8885c3c (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.020681 29329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43457aa7db604c8192ea4bc1f8885c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.020949 6769 ts_tablet_manager.cc:1888] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:13.021013 6769 log.cc:1196] T 089d9dc33a594712ac04ad28a0cd37b1 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/089d9dc33a594712ac04ad28a0cd37b1
I20240502 01:59:13.021724 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 089d9dc33a594712ac04ad28a0cd37b1 (table rebalancer_extra_table_28 [id=78f1e2dbafb84b93981e2e6a4b8362da]) successfully deleted
I20240502 01:59:13.022953 6775 tablet_bootstrap.cc:492] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:13.020458 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 3128a1fe3f5d49cfb334f1f1e3c6c147 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=70e0bc43e19d4b6e823d83b0e8de3bc4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.023303 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128a1fe3f5d49cfb334f1f1e3c6c147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.024370 6775 tablet_bootstrap.cc:654] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.020200 6771 tablet_bootstrap.cc:492] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:13.026479 6772 tablet_bootstrap.cc:492] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:13.026284 6774 tablet_bootstrap.cc:492] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:13.027348 6772 tablet_bootstrap.cc:654] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.027963 6774 tablet_bootstrap.cc:654] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.028613 6770 tablet_bootstrap.cc:492] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:13.028694 6770 ts_tablet_manager.cc:1367] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 01:59:13.028759 6775 tablet_bootstrap.cc:492] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:13.028828 6775 ts_tablet_manager.cc:1367] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:13.029043 6770 raft_consensus.cc:348] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.029141 6770 raft_consensus.cc:374] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.029172 6770 raft_consensus.cc:729] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.029249 6775 raft_consensus.cc:348] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.029314 6770 consensus_queue.cc:260] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.029362 6775 raft_consensus.cc:374] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.029395 6775 raft_consensus.cc:729] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.029503 6775 consensus_queue.cc:260] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.029676 6770 ts_tablet_manager.cc:1398] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.029819 6770 tablet_bootstrap.cc:492] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:13.029832 6775 ts_tablet_manager.cc:1398] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.030099 6775 tablet_bootstrap.cc:492] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:13.031015 6775 tablet_bootstrap.cc:654] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.031112 6770 tablet_bootstrap.cc:654] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.033473 6771 tablet_bootstrap.cc:654] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.034159 6770 tablet_bootstrap.cc:492] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:13.034278 6770 ts_tablet_manager.cc:1367] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:13.034746 6770 raft_consensus.cc:348] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.034855 6770 raft_consensus.cc:374] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.034934 6770 raft_consensus.cc:729] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.035063 6770 consensus_queue.cc:260] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.035588 6770 ts_tablet_manager.cc:1398] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:13.036044 6775 tablet_bootstrap.cc:492] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:13.036118 6775 ts_tablet_manager.cc:1367] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:13.036540 6775 raft_consensus.cc:348] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.036636 6775 raft_consensus.cc:374] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.036636 6771 tablet_bootstrap.cc:492] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:13.036664 6775 raft_consensus.cc:729] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.036728 6771 ts_tablet_manager.cc:1367] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:59:13.036757 6775 consensus_queue.cc:260] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.037019 6775 ts_tablet_manager.cc:1398] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.037137 6771 raft_consensus.cc:348] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.037317 6771 raft_consensus.cc:374] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.037387 6771 raft_consensus.cc:729] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.037555 6771 consensus_queue.cc:260] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.037782 6772 tablet_bootstrap.cc:492] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:13.037866 6772 ts_tablet_manager.cc:1367] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:13.038341 6772 raft_consensus.cc:348] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.038429 6772 raft_consensus.cc:374] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.038455 6772 raft_consensus.cc:729] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.038568 6772 consensus_queue.cc:260] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.038839 6772 ts_tablet_manager.cc:1398] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.039005 6772 tablet_bootstrap.cc:492] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:13.039888 6772 tablet_bootstrap.cc:654] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.040014 6771 ts_tablet_manager.cc:1398] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:13.040990 6772 tablet_bootstrap.cc:492] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:13.041074 6772 ts_tablet_manager.cc:1367] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:13.041481 6772 raft_consensus.cc:348] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.041761 6772 raft_consensus.cc:374] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.041836 6772 raft_consensus.cc:729] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.041983 6772 consensus_queue.cc:260] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.042395 6772 ts_tablet_manager.cc:1398] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.042600 6772 tablet_bootstrap.cc:492] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:13.043424 6772 tablet_bootstrap.cc:654] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.044407 6772 tablet_bootstrap.cc:492] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:13.044485 6772 ts_tablet_manager.cc:1367] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:13.044759 6774 tablet_bootstrap.cc:492] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:13.044842 6774 ts_tablet_manager.cc:1367] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 01:59:13.045006 6772 raft_consensus.cc:348] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.045078 6772 raft_consensus.cc:374] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.045095 6772 raft_consensus.cc:729] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.045181 6772 consensus_queue.cc:260] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.045303 6774 raft_consensus.cc:348] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.045403 6774 raft_consensus.cc:374] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.045421 6772 ts_tablet_manager.cc:1398] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.045430 6774 raft_consensus.cc:729] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.045557 6774 consensus_queue.cc:260] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.045848 6774 ts_tablet_manager.cc:1398] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.055709 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:13.059947 29079 raft_consensus.cc:1261] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:13.060295 27062 raft_consensus.cc:1261] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:13.060487 4649 consensus_queue.cc:1035] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.060878 4649 consensus_queue.cc:1035] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.064144 26506 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet b9ca883d28e147cc812ad22aa7ac9d73 (table rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98]). Expected version 1 got 0
I20240502 01:59:13.064654 26511 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet b9ca883d28e147cc812ad22aa7ac9d73 (table rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98]). Expected version 1 got 0
I20240502 01:59:13.066987 6799 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:13.067414 6799 tablet.cc:1620] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:13.067720 6794 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:13.068084 6794 tablet.cc:1620] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:13.068462 26511 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet b9ca883d28e147cc812ad22aa7ac9d73 (table rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98]). Expected version 1 got 0
I20240502 01:59:13.070592 6793 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:13.070890 6793 tablet.cc:1620] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:13.071660 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98] alter complete (version 1)
I20240502 01:59:13.125402 6694 raft_consensus.cc:482] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:13.125519 6694 raft_consensus.cc:504] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.125917 6694 leader_election.cc:288] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:13.126406 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0ffb725f2d843f8a03d8d0b711ce02d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:13.126546 26927 raft_consensus.cc:2454] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:13.126777 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0ffb725f2d843f8a03d8d0b711ce02d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:13.126911 29488 raft_consensus.cc:2454] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:13.127233 26730 leader_election.cc:302] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:13.127532 5896 raft_consensus.cc:2781] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:13.127599 5896 raft_consensus.cc:482] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:13.127651 5896 raft_consensus.cc:3037] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.128782 5896 raft_consensus.cc:504] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.129089 5896 leader_election.cc:288] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:13.129411 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0ffb725f2d843f8a03d8d0b711ce02d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:13.129451 29488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0ffb725f2d843f8a03d8d0b711ce02d" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:13.129528 26927 raft_consensus.cc:3037] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.129557 29488 raft_consensus.cc:3037] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.130561 26927 raft_consensus.cc:2454] T d0ffb725f2d843f8a03d8d0b711ce02d P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:13.130632 29488 raft_consensus.cc:2454] T d0ffb725f2d843f8a03d8d0b711ce02d P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:13.131978 26730 leader_election.cc:302] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:13.132236 5896 raft_consensus.cc:2781] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:13.132321 5896 raft_consensus.cc:686] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:13.132431 5896 consensus_queue.cc:237] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.139623 26508 catalog_manager.cc:5484] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:13.141831 6678 raft_consensus.cc:482] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:13.141925 6678 raft_consensus.cc:504] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.142341 6678 leader_election.cc:288] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:13.142767 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43457aa7db604c8192ea4bc1f8885c3c" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:13.142882 26793 raft_consensus.cc:2454] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:13.143091 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43457aa7db604c8192ea4bc1f8885c3c" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:13.143225 27062 raft_consensus.cc:2454] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:13.143486 29285 leader_election.cc:302] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:13.144162 6485 raft_consensus.cc:2781] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:13.144219 6485 raft_consensus.cc:482] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:13.144263 6485 raft_consensus.cc:3037] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.145309 6485 raft_consensus.cc:504] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.145640 6485 leader_election.cc:288] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:13.146126 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43457aa7db604c8192ea4bc1f8885c3c" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:13.146236 26793 raft_consensus.cc:3037] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.146318 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43457aa7db604c8192ea4bc1f8885c3c" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:13.146418 27062 raft_consensus.cc:3037] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.147195 26793 raft_consensus.cc:2454] T 43457aa7db604c8192ea4bc1f8885c3c P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:13.147490 27062 raft_consensus.cc:2454] T 43457aa7db604c8192ea4bc1f8885c3c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:13.147755 29286 leader_election.cc:302] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:13.148895 6677 raft_consensus.cc:2781] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:13.148983 6677 raft_consensus.cc:686] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:13.149107 6677 consensus_queue.cc:237] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:13.151818 26515 catalog_manager.cc:5484] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:13.176661 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:13.182018 6793 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:13.182356 6793 tablet.cc:1620] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:13.182689 6799 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:13.182760 6794 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:13.182945 6799 tablet.cc:1620] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:13.183022 6794 tablet.cc:1620] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:13.187762 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=626280a69dac4664a65a50a93ac35f98] alter complete (version 2)
I20240502 01:59:13.289309 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:13.293934 6793 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:13.295670 6794 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:13.297114 6799 tablet.cc:1722] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:13.302516 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=626280a69dac4664a65a50a93ac35f98] alter complete (version 3)
I20240502 01:59:13.395958 6694 raft_consensus.cc:482] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:13.396080 6694 raft_consensus.cc:504] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.396485 6694 leader_election.cc:288] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:13.396730 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128a1fe3f5d49cfb334f1f1e3c6c147" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:13.396862 29349 raft_consensus.cc:2454] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:13.397102 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128a1fe3f5d49cfb334f1f1e3c6c147" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:13.397233 26927 raft_consensus.cc:2454] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:13.397508 26730 leader_election.cc:302] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:13.397771 6694 raft_consensus.cc:2781] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:13.397825 6694 raft_consensus.cc:482] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:13.397856 6694 raft_consensus.cc:3037] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.398918 6694 raft_consensus.cc:504] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.399482 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128a1fe3f5d49cfb334f1f1e3c6c147" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:13.399495 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128a1fe3f5d49cfb334f1f1e3c6c147" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:13.399580 29349 raft_consensus.cc:3037] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.399601 26927 raft_consensus.cc:3037] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.400396 29349 raft_consensus.cc:2454] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:13.400480 26927 raft_consensus.cc:2454] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:13.400645 26727 leader_election.cc:302] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:13.400816 5896 raft_consensus.cc:2781] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:13.400839 6694 leader_election.cc:288] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:13.400959 5896 raft_consensus.cc:686] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:13.401118 5896 consensus_queue.cc:237] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:13.403457 26515 catalog_manager.cc:5484] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:13.406103 26515 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:13.406215 26515 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:13.407788 26515 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet b9ca883d28e147cc812ad22aa7ac9d73
I20240502 01:59:13.408313 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ca883d28e147cc812ad22aa7ac9d73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:13 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:13.408396 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ca883d28e147cc812ad22aa7ac9d73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:13 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:13.408564 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ca883d28e147cc812ad22aa7ac9d73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:13 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:13.408603 6830 tablet_replica.cc:330] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:13.408694 6831 tablet_replica.cc:330] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:13.408736 6830 raft_consensus.cc:2229] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:13.408835 6831 raft_consensus.cc:2229] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:13.408869 6832 tablet_replica.cc:330] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:13.408910 6830 raft_consensus.cc:2258] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:13.409060 6831 raft_consensus.cc:2258] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:13.409062 6832 raft_consensus.cc:2229] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:13.409184 6832 raft_consensus.cc:2258] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:13.409736 6830 ts_tablet_manager.cc:1875] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:13.409778 6832 ts_tablet_manager.cc:1875] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:13.409834 6831 ts_tablet_manager.cc:1875] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:13.411712 6832 ts_tablet_manager.cc:1888] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:13.411782 6832 log.cc:1196] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/b9ca883d28e147cc812ad22aa7ac9d73
I20240502 01:59:13.412030 6830 ts_tablet_manager.cc:1888] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:13.412078 6832 ts_tablet_manager.cc:1909] T b9ca883d28e147cc812ad22aa7ac9d73 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:13.412109 6830 log.cc:1196] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/b9ca883d28e147cc812ad22aa7ac9d73
I20240502 01:59:13.412393 6830 ts_tablet_manager.cc:1909] T b9ca883d28e147cc812ad22aa7ac9d73 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:13.412773 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet b9ca883d28e147cc812ad22aa7ac9d73 (table rebalancer_dynamic_table_new_name [id=626280a69dac4664a65a50a93ac35f98]) successfully deleted
I20240502 01:59:13.413048 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet b9ca883d28e147cc812ad22aa7ac9d73 (table rebalancer_dynamic_table_new_name [id=626280a69dac4664a65a50a93ac35f98]) successfully deleted
I20240502 01:59:13.414765 6831 ts_tablet_manager.cc:1888] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:13.414834 6831 log.cc:1196] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/b9ca883d28e147cc812ad22aa7ac9d73
I20240502 01:59:13.415072 6831 ts_tablet_manager.cc:1909] T b9ca883d28e147cc812ad22aa7ac9d73 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:13.415608 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet b9ca883d28e147cc812ad22aa7ac9d73 (table rebalancer_dynamic_table_new_name [id=626280a69dac4664a65a50a93ac35f98]) successfully deleted
I20240502 01:59:13.430572 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:13.437134 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:13.442595 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:13.445204 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:13.454622 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:13.467424 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:13.467865 29059 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 01:59:13.470660 26774 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:13.509653 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:13.515184 26641 tablet_service.cc:1467] Processing CreateTablet for tablet c0067f4331b040898ddd6c5936c42bf7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]), partition=
I20240502 01:59:13.515205 29059 tablet_service.cc:1467] Processing CreateTablet for tablet c0067f4331b040898ddd6c5936c42bf7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]), partition=
I20240502 01:59:13.515440 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0067f4331b040898ddd6c5936c42bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.515440 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0067f4331b040898ddd6c5936c42bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.517275 6860 tablet_bootstrap.cc:492] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:13.518220 6860 tablet_bootstrap.cc:654] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.518906 29196 tablet_service.cc:1467] Processing CreateTablet for tablet c0067f4331b040898ddd6c5936c42bf7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]), partition=
I20240502 01:59:13.519124 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0067f4331b040898ddd6c5936c42bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.519341 6860 tablet_bootstrap.cc:492] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:13.519378 6861 tablet_bootstrap.cc:492] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:13.519418 6860 ts_tablet_manager.cc:1367] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:13.519817 6860 raft_consensus.cc:348] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.519917 6860 raft_consensus.cc:374] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.519945 6860 raft_consensus.cc:729] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.520051 6860 consensus_queue.cc:260] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.520130 6861 tablet_bootstrap.cc:654] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.520349 6860 ts_tablet_manager.cc:1398] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:13.521154 6861 tablet_bootstrap.cc:492] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:13.521287 6864 tablet_bootstrap.cc:492] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:13.521231 6861 ts_tablet_manager.cc:1367] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:13.521965 6861 raft_consensus.cc:348] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.522209 6861 raft_consensus.cc:374] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.522262 6861 raft_consensus.cc:729] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.522655 6861 consensus_queue.cc:260] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.523819 6864 tablet_bootstrap.cc:654] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.524873 6864 tablet_bootstrap.cc:492] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:13.524983 6864 ts_tablet_manager.cc:1367] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:13.525462 6864 raft_consensus.cc:348] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.525641 6864 raft_consensus.cc:374] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:13.525724 6864 raft_consensus.cc:729] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:13.525847 6861 ts_tablet_manager.cc:1398] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:13.525882 6864 consensus_queue.cc:260] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.527086 6864 ts_tablet_manager.cc:1398] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:13.534766 29213 consensus_queue.cc:237] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:13.536032 26793 raft_consensus.cc:1261] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:13.536088 29079 raft_consensus.cc:1261] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:13.536392 5818 consensus_queue.cc:1035] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:13.536695 5818 consensus_queue.cc:1035] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:13.538332 5818 raft_consensus.cc:2932] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:13.538617 29079 raft_consensus.cc:2932] T 499d2ca7bffc42748591db5280f829f2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:13.538735 26793 raft_consensus.cc:2932] T 499d2ca7bffc42748591db5280f829f2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:13.542012 26515 catalog_manager.cc:5484] T 499d2ca7bffc42748591db5280f829f2 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:13.543922 6328 raft_consensus.cc:482] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:13.544003 6328 raft_consensus.cc:504] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.544345 6328 leader_election.cc:288] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:13.546259 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0067f4331b040898ddd6c5936c42bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:13.546254 29328 tablet_service.cc:1514] Processing DeleteTablet for tablet 499d2ca7bffc42748591db5280f829f2 with delete_type TABLET_DATA_TOMBSTONED (TS 468dd27456c64d67b22e8a58af93d93d not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:13.546378 29216 raft_consensus.cc:2454] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:13.546626 6878 tablet_replica.cc:330] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:13.546733 29015 leader_election.cc:302] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:13.546877 5443 raft_consensus.cc:2781] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:13.546938 5443 raft_consensus.cc:482] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:13.546974 5443 raft_consensus.cc:3037] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.547072 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0067f4331b040898ddd6c5936c42bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:13.547209 26659 raft_consensus.cc:2454] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:13.547956 5443 raft_consensus.cc:504] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.548261 5443 leader_election.cc:288] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:13.548467 6878 raft_consensus.cc:2229] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:13.548718 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0067f4331b040898ddd6c5936c42bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:13.548825 29216 raft_consensus.cc:3037] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.549002 6878 raft_consensus.cc:2258] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:13.549417 6878 ts_tablet_manager.cc:1875] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:13.549736 29216 raft_consensus.cc:2454] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:13.550230 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0067f4331b040898ddd6c5936c42bf7" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:13.550334 26659 raft_consensus.cc:3037] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:13.551326 26659 raft_consensus.cc:2454] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:13.551609 6878 ts_tablet_manager.cc:1888] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:13.551631 29015 leader_election.cc:302] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:13.551780 5443 raft_consensus.cc:2781] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:13.551847 5443 raft_consensus.cc:686] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:13.551952 5443 consensus_queue.cc:237] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:13.551719 6878 log.cc:1196] T 499d2ca7bffc42748591db5280f829f2 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/499d2ca7bffc42748591db5280f829f2
I20240502 01:59:13.558188 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 499d2ca7bffc42748591db5280f829f2 (table rebalancer_extra_table_30 [id=5ca1c45fce8342d882bce9c9775c9edb]) successfully deleted
I20240502 01:59:13.559975 26507 catalog_manager.cc:5484] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:13.588953 6678 consensus_queue.cc:1035] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.591539 6678 consensus_queue.cc:1035] T 43457aa7db604c8192ea4bc1f8885c3c P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.597035 5896 consensus_queue.cc:1035] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.599903 6828 consensus_queue.cc:1035] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.667924 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:13.671306 29213 raft_consensus.cc:1261] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:13.671350 26659 raft_consensus.cc:1261] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:13.671648 5443 consensus_queue.cc:1035] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.671924 5443 consensus_queue.cc:1035] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.675467 26507 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]). Expected version 1 got 0
I20240502 01:59:13.675773 26508 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]). Expected version 1 got 0
I20240502 01:59:13.675894 6799 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:13.676188 6799 tablet.cc:1620] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:13.677318 6892 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:13.677660 6892 tablet.cc:1620] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:13.677963 6793 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:13.678270 6793 tablet.cc:1620] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:13.680091 26508 catalog_manager.cc:5576] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929) does not have the latest schema for tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]). Expected version 1 got 0
I20240502 01:59:13.681054 26507 catalog_manager.cc:5576] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103) does not have the latest schema for tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb]). Expected version 1 got 0
I20240502 01:59:13.683288 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb] alter complete (version 1)
I20240502 01:59:13.785037 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:13.791420 6799 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:13.791739 6799 tablet.cc:1620] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:13.792161 6793 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:13.792474 6793 tablet.cc:1620] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: MemRowSet was empty: no flush needed.
I20240502 01:59:13.793087 6892 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:13.793390 6892 tablet.cc:1620] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: MemRowSet was empty: no flush needed.
I20240502 01:59:13.798691 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=a32e8f15b642430d884aee3fcc8e0afb] alter complete (version 2)
I20240502 01:59:13.890333 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:13.892550 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:13.893431 29468 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:13.902060 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:13.902715 26515 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:13.910612 6799 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:13.914083 26641 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:13.914211 6793 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:13.919903 29059 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:13.921427 26507 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=a32e8f15b642430d884aee3fcc8e0afb] alter complete (version 3)
I20240502 01:59:13.924150 26774 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:13.924706 6892 tablet.cc:1722] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:13.953704 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:13.983599 5896 consensus_queue.cc:1035] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:13.985960 26515 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:13.994998 29328 tablet_service.cc:1467] Processing CreateTablet for tablet c4a7be37454045528f9126385ff48160 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.995301 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a7be37454045528f9126385ff48160. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.995309 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 7d6c35112da74b34b7c71318c61fda5e (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.995558 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d6c35112da74b34b7c71318c61fda5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.995582 29058 tablet_service.cc:1467] Processing CreateTablet for tablet 714bb06a8a56413f8210f86c1fa9f717 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.996253 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714bb06a8a56413f8210f86c1fa9f717. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.997618 6861 tablet_bootstrap.cc:492] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:13.997822 26908 tablet_service.cc:1467] Processing CreateTablet for tablet 714bb06a8a56413f8210f86c1fa9f717 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:13.998078 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714bb06a8a56413f8210f86c1fa9f717. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:13.998559 6861 tablet_bootstrap.cc:654] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:13.998819 6946 tablet_bootstrap.cc:492] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:13.999716 6946 tablet_bootstrap.cc:654] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.000198 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 7d6c35112da74b34b7c71318c61fda5e (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.000248 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 714bb06a8a56413f8210f86c1fa9f717 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.000515 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d6c35112da74b34b7c71318c61fda5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.000550 29454 tablet_service.cc:1467] Processing CreateTablet for tablet c4a7be37454045528f9126385ff48160 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.000757 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a7be37454045528f9126385ff48160. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.000694 29196 tablet_service.cc:1467] Processing CreateTablet for tablet 7d6c35112da74b34b7c71318c61fda5e (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.000926 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d6c35112da74b34b7c71318c61fda5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.001015 6946 tablet_bootstrap.cc:492] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:14.001108 6946 ts_tablet_manager.cc:1367] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:14.001506 6946 raft_consensus.cc:348] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.001600 6946 raft_consensus.cc:374] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.001628 6946 raft_consensus.cc:729] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.001737 6946 consensus_queue.cc:260] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.002060 6946 ts_tablet_manager.cc:1398] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.002732 6950 tablet_bootstrap.cc:492] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:14.003008 6864 tablet_bootstrap.cc:492] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:14.003858 6864 tablet_bootstrap.cc:654] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.003909 6950 tablet_bootstrap.cc:654] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.004137 6948 tablet_bootstrap.cc:492] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:14.004906 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714bb06a8a56413f8210f86c1fa9f717. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.006903 6948 tablet_bootstrap.cc:654] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.007403 6828 consensus_queue.cc:1035] T 3128a1fe3f5d49cfb334f1f1e3c6c147 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.007535 26641 tablet_service.cc:1467] Processing CreateTablet for tablet c4a7be37454045528f9126385ff48160 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=fcd755ce62b540ecbeaa56fc0995423b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.007794 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a7be37454045528f9126385ff48160. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.011569 6860 tablet_bootstrap.cc:492] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:14.011868 6864 tablet_bootstrap.cc:492] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:14.011943 6864 ts_tablet_manager.cc:1367] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 01:59:14.012445 6860 tablet_bootstrap.cc:654] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.012396 6864 raft_consensus.cc:348] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.012488 6864 raft_consensus.cc:374] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.012516 6864 raft_consensus.cc:729] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.012639 6864 consensus_queue.cc:260] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.012674 6861 tablet_bootstrap.cc:492] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:14.012749 6861 ts_tablet_manager.cc:1367] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:59:14.012921 6864 ts_tablet_manager.cc:1398] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.013164 6861 raft_consensus.cc:348] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.013257 6861 raft_consensus.cc:374] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.013295 6861 raft_consensus.cc:729] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.013401 6861 consensus_queue.cc:260] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.013676 6861 ts_tablet_manager.cc:1398] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:14.013825 6861 tablet_bootstrap.cc:492] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:14.014724 6861 tablet_bootstrap.cc:654] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.015483 6860 tablet_bootstrap.cc:492] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:14.015560 6860 ts_tablet_manager.cc:1367] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:14.015996 6860 raft_consensus.cc:348] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.016093 6860 raft_consensus.cc:374] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.016126 6860 raft_consensus.cc:729] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.016234 6860 consensus_queue.cc:260] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.016502 6860 ts_tablet_manager.cc:1398] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.019170 6950 tablet_bootstrap.cc:492] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:14.019325 6950 ts_tablet_manager.cc:1367] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:59:14.020063 6950 raft_consensus.cc:348] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.021332 6950 raft_consensus.cc:374] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.021466 6950 raft_consensus.cc:729] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.021553 6861 tablet_bootstrap.cc:492] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:14.021622 6861 ts_tablet_manager.cc:1367] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:59:14.021631 6950 consensus_queue.cc:260] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.021925 6950 ts_tablet_manager.cc:1398] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:14.022007 6861 raft_consensus.cc:348] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.022104 6950 tablet_bootstrap.cc:492] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:14.022131 6861 raft_consensus.cc:374] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.022159 6861 raft_consensus.cc:729] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.022262 6861 consensus_queue.cc:260] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.022509 6861 ts_tablet_manager.cc:1398] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:14.023100 6950 tablet_bootstrap.cc:654] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.023430 26507 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:14.023535 26507 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:14.025070 6948 tablet_bootstrap.cc:492] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:14.026059 6948 ts_tablet_manager.cc:1367] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 01:59:14.026866 26507 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet c0067f4331b040898ddd6c5936c42bf7
I20240502 01:59:14.027308 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet c0067f4331b040898ddd6c5936c42bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:14 UTC) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:14.027264 6948 raft_consensus.cc:348] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.027333 6950 tablet_bootstrap.cc:492] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:14.027366 6948 raft_consensus.cc:374] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.027408 6950 ts_tablet_manager.cc:1367] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 01:59:14.027396 6948 raft_consensus.cc:729] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.027580 6948 consensus_queue.cc:260] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.027681 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet c0067f4331b040898ddd6c5936c42bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:14 UTC) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:14.027849 6948 ts_tablet_manager.cc:1398] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.027828 6950 raft_consensus.cc:348] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.027925 6950 raft_consensus.cc:374] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.027956 6950 raft_consensus.cc:729] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.028056 6950 consensus_queue.cc:260] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.028347 6950 ts_tablet_manager.cc:1398] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:14.028339 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet c0067f4331b040898ddd6c5936c42bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:14 UTC) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:14.028506 6950 tablet_bootstrap.cc:492] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:14.029250 6950 tablet_bootstrap.cc:654] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.030189 6964 tablet_replica.cc:330] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:14.032642 6965 tablet_replica.cc:330] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:14.032824 6965 raft_consensus.cc:2229] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:14.033043 6965 raft_consensus.cc:2258] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.033250 6950 tablet_bootstrap.cc:492] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:14.033334 6950 ts_tablet_manager.cc:1367] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 01:59:14.033592 6965 ts_tablet_manager.cc:1875] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:14.034479 6964 raft_consensus.cc:2229] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.034847 6964 raft_consensus.cc:2258] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.034873 6950 raft_consensus.cc:348] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.034967 6950 raft_consensus.cc:374] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.034997 6950 raft_consensus.cc:729] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.035091 6950 consensus_queue.cc:260] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.035354 6950 ts_tablet_manager.cc:1398] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:14.035548 6964 ts_tablet_manager.cc:1875] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:14.035748 6966 tablet_replica.cc:330] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:14.035917 6966 raft_consensus.cc:2229] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.037362 6965 ts_tablet_manager.cc:1888] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:14.037767 6966 raft_consensus.cc:2258] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.037842 6965 log.cc:1196] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/c0067f4331b040898ddd6c5936c42bf7
I20240502 01:59:14.038136 6965 ts_tablet_manager.cc:1909] T c0067f4331b040898ddd6c5936c42bf7 P f3d18a019013487ab801599cfb80e4b4: Deleting consensus metadata
I20240502 01:59:14.038738 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table_new_name [id=a32e8f15b642430d884aee3fcc8e0afb]) successfully deleted
I20240502 01:59:14.039458 6966 ts_tablet_manager.cc:1875] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:14.042188 6966 ts_tablet_manager.cc:1888] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:14.042265 6966 log.cc:1196] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/c0067f4331b040898ddd6c5936c42bf7
I20240502 01:59:14.042554 6966 ts_tablet_manager.cc:1909] T c0067f4331b040898ddd6c5936c42bf7 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting consensus metadata
I20240502 01:59:14.042652 6964 ts_tablet_manager.cc:1888] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:14.042717 6964 log.cc:1196] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0067f4331b040898ddd6c5936c42bf7
I20240502 01:59:14.043016 6964 ts_tablet_manager.cc:1909] T c0067f4331b040898ddd6c5936c42bf7 P 998b57e2badc4d779c586c6579149114: Deleting consensus metadata
I20240502 01:59:14.043465 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table_new_name [id=a32e8f15b642430d884aee3fcc8e0afb]) successfully deleted
I20240502 01:59:14.043695 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet c0067f4331b040898ddd6c5936c42bf7 (table rebalancer_dynamic_table_new_name [id=a32e8f15b642430d884aee3fcc8e0afb]) successfully deleted
I20240502 01:59:14.047083 27062 consensus_queue.cc:237] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.048238 29079 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:14.048243 29213 raft_consensus.cc:1261] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:14.048702 6373 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:14.048882 4345 consensus_queue.cc:1035] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:14.050531 6373 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.050884 29213 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.051112 29079 raft_consensus.cc:2932] T e1950ace91824415b9bf3d1657bad72f P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.053632 26507 catalog_manager.cc:5484] T e1950ace91824415b9bf3d1657bad72f P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.056638 26908 tablet_service.cc:1514] Processing DeleteTablet for tablet e1950ace91824415b9bf3d1657bad72f with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:14.056990 6988 tablet_replica.cc:330] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:14.057180 6988 raft_consensus.cc:2229] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.057420 6988 raft_consensus.cc:2258] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.058411 6988 ts_tablet_manager.cc:1875] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:14.061971 6988 ts_tablet_manager.cc:1888] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 01:59:14.062053 6988 log.cc:1196] T e1950ace91824415b9bf3d1657bad72f P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/e1950ace91824415b9bf3d1657bad72f
I20240502 01:59:14.062558 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet e1950ace91824415b9bf3d1657bad72f (table rebalance_test_table_13 [id=d7e58e10bc3742c694eee2f14db77982]) successfully deleted
I20240502 01:59:14.131207 26507 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:14.140340 5443 raft_consensus.cc:482] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:14.140434 5443 raft_consensus.cc:504] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.140782 5443 leader_election.cc:288] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:14.141619 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714bb06a8a56413f8210f86c1fa9f717" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:14.141734 29489 raft_consensus.cc:2454] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:14.142349 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef6f9219f0d4f1884b948a07737a9f0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20]), partition=
I20240502 01:59:14.142565 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef6f9219f0d4f1884b948a07737a9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.144538 6946 tablet_bootstrap.cc:492] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:14.145364 6946 tablet_bootstrap.cc:654] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.146075 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef6f9219f0d4f1884b948a07737a9f0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20]), partition=
I20240502 01:59:14.146318 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef6f9219f0d4f1884b948a07737a9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.147341 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714bb06a8a56413f8210f86c1fa9f717" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:14.147461 26927 raft_consensus.cc:2454] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 0.
I20240502 01:59:14.148337 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 9ef6f9219f0d4f1884b948a07737a9f0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20]), partition=
I20240502 01:59:14.148468 29016 leader_election.cc:302] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:14.148557 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ef6f9219f0d4f1884b948a07737a9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.148715 5443 raft_consensus.cc:2781] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:14.148767 5443 raft_consensus.cc:482] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:14.148803 5443 raft_consensus.cc:3037] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.149776 5443 raft_consensus.cc:504] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.151181 5443 leader_election.cc:288] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:14.151669 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714bb06a8a56413f8210f86c1fa9f717" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:14.151772 29489 raft_consensus.cc:3037] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.152630 29489 raft_consensus.cc:2454] T 714bb06a8a56413f8210f86c1fa9f717 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:14.153241 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714bb06a8a56413f8210f86c1fa9f717" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:14.153344 26927 raft_consensus.cc:3037] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.153806 7028 tablet_bootstrap.cc:492] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:14.154322 26927 raft_consensus.cc:2454] T 714bb06a8a56413f8210f86c1fa9f717 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:14.154908 29016 leader_election.cc:302] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:14.155052 5443 raft_consensus.cc:2781] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:14.155134 5443 raft_consensus.cc:686] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:14.155251 5443 consensus_queue.cc:237] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:14.155845 7028 tablet_bootstrap.cc:654] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.158367 26514 catalog_manager.cc:5484] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:14.160348 6946 tablet_bootstrap.cc:492] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:14.160424 6946 ts_tablet_manager.cc:1367] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 01:59:14.160848 6946 raft_consensus.cc:348] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.160933 6946 raft_consensus.cc:374] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.160959 6946 raft_consensus.cc:729] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.161065 6946 consensus_queue.cc:260] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.161357 6946 ts_tablet_manager.cc:1398] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.168179 7027 tablet_bootstrap.cc:492] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:14.169652 7027 tablet_bootstrap.cc:654] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.171432 7028 tablet_bootstrap.cc:492] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:14.171526 7028 ts_tablet_manager.cc:1367] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 01:59:14.171988 7028 raft_consensus.cc:348] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.172089 7028 raft_consensus.cc:374] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.172122 7028 raft_consensus.cc:729] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.172230 7028 consensus_queue.cc:260] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.172509 7028 ts_tablet_manager.cc:1398] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.177129 7027 tablet_bootstrap.cc:492] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:14.177268 7027 ts_tablet_manager.cc:1367] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240502 01:59:14.177788 7027 raft_consensus.cc:348] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.177963 7027 raft_consensus.cc:374] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.178066 7027 raft_consensus.cc:729] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.178262 7027 consensus_queue.cc:260] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.179597 7027 ts_tablet_manager.cc:1398] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:14.194401 5780 raft_consensus.cc:482] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:14.194499 5780 raft_consensus.cc:504] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.194911 5780 leader_election.cc:288] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:14.195151 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:14.195286 29079 raft_consensus.cc:2454] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:14.195582 29419 leader_election.cc:302] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:14.195747 5780 raft_consensus.cc:2781] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:14.195736 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:14.195804 5780 raft_consensus.cc:482] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:14.195839 5780 raft_consensus.cc:3037] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.195868 29216 raft_consensus.cc:2454] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:14.196913 5780 raft_consensus.cc:504] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.197242 5780 leader_election.cc:288] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:14.197506 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:14.197530 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:14.197605 29079 raft_consensus.cc:3037] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.197643 29216 raft_consensus.cc:3037] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.200556 29216 raft_consensus.cc:2454] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:14.200660 29079 raft_consensus.cc:2454] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:14.200927 29421 leader_election.cc:302] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:14.201113 5780 raft_consensus.cc:2781] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:14.201195 5780 raft_consensus.cc:686] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:14.201319 5780 consensus_queue.cc:237] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.204075 26508 catalog_manager.cc:5484] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.248121 4345 raft_consensus.cc:482] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:14.248241 4345 raft_consensus.cc:504] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.248700 4345 leader_election.cc:288] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:14.249035 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef6f9219f0d4f1884b948a07737a9f0" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:14.249143 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef6f9219f0d4f1884b948a07737a9f0" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:14.249183 29348 raft_consensus.cc:2454] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:14.249279 26793 raft_consensus.cc:2454] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 0.
I20240502 01:59:14.249495 26995 leader_election.cc:302] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:14.249712 4345 raft_consensus.cc:2781] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:14.249771 4345 raft_consensus.cc:482] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:14.249810 4345 raft_consensus.cc:3037] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.250844 4345 raft_consensus.cc:504] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.251428 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef6f9219f0d4f1884b948a07737a9f0" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:14.251518 29348 raft_consensus.cc:3037] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.251489 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ef6f9219f0d4f1884b948a07737a9f0" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:14.251591 26793 raft_consensus.cc:3037] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.252287 29348 raft_consensus.cc:2454] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:14.252367 26793 raft_consensus.cc:2454] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68e5b1eeb04448fea818a14991eab87c in term 1.
I20240502 01:59:14.252548 26995 leader_election.cc:302] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:14.252696 6980 raft_consensus.cc:2781] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:14.252816 6980 raft_consensus.cc:686] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Becoming Leader. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Running, Role: LEADER
I20240502 01:59:14.251206 4345 leader_election.cc:288] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:14.252969 6980 consensus_queue.cc:237] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.253952 6865 raft_consensus.cc:482] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:14.254071 6865 raft_consensus.cc:504] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.254443 6865 leader_election.cc:288] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:14.254583 5771 raft_consensus.cc:482] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:14.254678 5771 raft_consensus.cc:504] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.255065 5771 leader_election.cc:288] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:14.255252 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:14.255378 29489 raft_consensus.cc:2454] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:14.255475 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:14.255579 26659 raft_consensus.cc:2454] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:14.255568 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:14.255661 29349 raft_consensus.cc:2454] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:14.255849 29421 leader_election.cc:302] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:14.255898 26595 leader_election.cc:302] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:14.256067 6865 raft_consensus.cc:2781] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:14.256122 6865 raft_consensus.cc:482] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:14.256158 6865 raft_consensus.cc:3037] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.256139 5771 raft_consensus.cc:2781] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:14.256188 5771 raft_consensus.cc:482] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:14.256244 5771 raft_consensus.cc:3037] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.257170 6865 raft_consensus.cc:504] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.257171 5771 raft_consensus.cc:504] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.257494 6865 leader_election.cc:288] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:14.257496 5771 leader_election.cc:288] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:14.257925 29349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:14.258049 29349 raft_consensus.cc:3037] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.258086 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:14.258087 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:14.258212 29489 raft_consensus.cc:2379] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 998b57e2badc4d779c586c6579149114 in current term 1: Already voted for candidate 7c693d0933c64d9babc07f404df45cd1 in this term.
I20240502 01:59:14.258412 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:14.258769 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7be37454045528f9126385ff48160" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:14.258903 26659 raft_consensus.cc:2379] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c693d0933c64d9babc07f404df45cd1 in current term 1: Already voted for candidate 998b57e2badc4d779c586c6579149114 in this term.
I20240502 01:59:14.258890 29349 raft_consensus.cc:2454] T c4a7be37454045528f9126385ff48160 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:14.259159 26592 leader_election.cc:302] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:14.259369 6865 raft_consensus.cc:2781] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:14.259375 29421 leader_election.cc:302] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1; no voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114
I20240502 01:59:14.259502 6865 raft_consensus.cc:686] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:14.259629 6865 consensus_queue.cc:237] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.259693 5771 raft_consensus.cc:2726] T c4a7be37454045528f9126385ff48160 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 01:59:14.260895 26507 catalog_manager.cc:5484] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: term changed from 0 to 1, leader changed from <none> to 68e5b1eeb04448fea818a14991eab87c (127.19.43.196). New cstate: current_term: 1 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:14.261957 26515 catalog_manager.cc:5484] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:14.283043 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:14.289980 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:14.290294 26641 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:14.296279 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:14.311256 26774 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:14.312554 27043 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:14.313427 29451 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:14.314905 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:14.377353 29489 consensus_queue.cc:237] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:14.378659 29079 raft_consensus.cc:1261] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:14.378680 27062 raft_consensus.cc:1261] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:14.379050 5780 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:14.379240 5771 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:14.381088 5780 raft_consensus.cc:2932] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:14.381462 27062 raft_consensus.cc:2932] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:14.384461 26515 catalog_manager.cc:5484] T 9ca8563ff68c4fa590c17ef58e67afa0 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:14.387720 29328 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca8563ff68c4fa590c17ef58e67afa0 with delete_type TABLET_DATA_TOMBSTONED (TS 468dd27456c64d67b22e8a58af93d93d not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:14.388065 7066 tablet_replica.cc:330] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:14.388262 7066 raft_consensus.cc:2229] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.388450 7066 raft_consensus.cc:2258] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.388939 7066 ts_tablet_manager.cc:1875] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:14.391144 7066 ts_tablet_manager.cc:1888] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:14.391234 7066 log.cc:1196] T 9ca8563ff68c4fa590c17ef58e67afa0 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/9ca8563ff68c4fa590c17ef58e67afa0
I20240502 01:59:14.391243 29079 raft_consensus.cc:2932] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:14.392628 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 9ca8563ff68c4fa590c17ef58e67afa0 (table rebalancer_extra_table_38 [id=4504e908fd324f648d1e375bbdbed153]) successfully deleted
I20240502 01:59:14.392956 26507 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:14.398221 26793 raft_consensus.cc:1261] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:14.398414 29349 raft_consensus.cc:1261] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:14.398635 6980 consensus_queue.cc:1035] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.400067 6980 consensus_queue.cc:1035] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.402242 7079 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:14.402449 7081 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:14.402786 26507 catalog_manager.cc:5576] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007) does not have the latest schema for tablet 9ef6f9219f0d4f1884b948a07737a9f0 (table rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20]). Expected version 1 got 0
I20240502 01:59:14.402798 7081 tablet.cc:1620] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:14.403519 7079 tablet.cc:1620] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:14.405936 26508 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 9ef6f9219f0d4f1884b948a07737a9f0 (table rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20]). Expected version 1 got 0
I20240502 01:59:14.410228 7083 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:14.411485 7083 tablet.cc:1620] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:14.412109 26506 catalog_manager.cc:5576] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303) does not have the latest schema for tablet 9ef6f9219f0d4f1884b948a07737a9f0 (table rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20]). Expected version 1 got 0
I20240502 01:59:14.412693 26515 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20] alter complete (version 1)
I20240502 01:59:14.514638 29451 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:14.516119 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:14.520391 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:14.525122 7079 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:14.525440 7079 tablet.cc:1620] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:14.526314 7081 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:14.526572 7081 tablet.cc:1620] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:14.527591 27043 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:14.531749 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=dc57ac1188164725a3e7178ad7dbbd20] alter complete (version 2)
I20240502 01:59:14.533967 7083 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:14.534289 7083 tablet.cc:1620] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:14.541182 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:14.545040 26774 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:14.550817 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:14.560595 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:14.560855 26641 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:14.606837 5443 consensus_queue.cc:1035] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.616784 6891 consensus_queue.cc:1035] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.630774 5771 consensus_queue.cc:1035] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.633133 26508 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:14.634799 5780 consensus_queue.cc:1035] T 7d6c35112da74b34b7c71318c61fda5e P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.638734 7079 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:14.639305 7083 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:14.643844 7081 tablet.cc:1722] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:14.645771 26793 consensus_queue.cc:237] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.646919 29216 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:14.647259 29079 raft_consensus.cc:1261] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:14.647321 6953 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:14.648267 6953 consensus_queue.cc:1035] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:14.649322 26510 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=dc57ac1188164725a3e7178ad7dbbd20] alter complete (version 3)
I20240502 01:59:14.649411 6828 raft_consensus.cc:2932] T d216b8ccd8ea43df85c1013cdaf36564 P b290c1bf893444078ed721f25995d124 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.649717 29216 raft_consensus.cc:2932] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.652524 26515 catalog_manager.cc:5484] T d216b8ccd8ea43df85c1013cdaf36564 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index 2 to 4, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New cstate: current_term: 2 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.653198 29079 raft_consensus.cc:2932] T d216b8ccd8ea43df85c1013cdaf36564 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:14.656920 26641 tablet_service.cc:1514] Processing DeleteTablet for tablet d216b8ccd8ea43df85c1013cdaf36564 with delete_type TABLET_DATA_TOMBSTONED (TS 998b57e2badc4d779c586c6579149114 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56476
I20240502 01:59:14.657594 7154 tablet_replica.cc:330] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: stopping tablet replica
I20240502 01:59:14.657795 7154 raft_consensus.cc:2229] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.658111 7154 raft_consensus.cc:2258] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.658475 7154 ts_tablet_manager.cc:1875] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:14.660274 7154 ts_tablet_manager.cc:1888] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240502 01:59:14.660336 7154 log.cc:1196] T d216b8ccd8ea43df85c1013cdaf36564 P 998b57e2badc4d779c586c6579149114: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/wals/d216b8ccd8ea43df85c1013cdaf36564
I20240502 01:59:14.660848 26501 catalog_manager.cc:4830] TS 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): tablet d216b8ccd8ea43df85c1013cdaf36564 (table rebalancer_extra_table_26 [id=f8ab9d33449b4772a94f39d6680cd5f6]) successfully deleted
I20240502 01:59:14.697701 6865 consensus_queue.cc:1035] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.701921 6386 consensus_queue.cc:1035] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:14.752590 26509 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:14.752722 26509 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:14.764883 26509 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 9ef6f9219f0d4f1884b948a07737a9f0
I20240502 01:59:14.765686 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ef6f9219f0d4f1884b948a07737a9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:14 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:14.766111 7207 tablet_replica.cc:330] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:14.766356 7207 raft_consensus.cc:2229] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.766546 7207 raft_consensus.cc:2258] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.766692 29328 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ef6f9219f0d4f1884b948a07737a9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:14 UTC) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:14.766862 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ef6f9219f0d4f1884b948a07737a9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:14 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:14.767153 7208 tablet_replica.cc:330] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:14.767180 7066 tablet_replica.cc:330] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:14.767263 7207 ts_tablet_manager.cc:1875] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:14.767325 7066 raft_consensus.cc:2229] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.767398 7208 raft_consensus.cc:2229] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:14.767439 7066 raft_consensus.cc:2258] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.767565 7208 raft_consensus.cc:2258] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.768121 7208 ts_tablet_manager.cc:1875] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:14.768170 7066 ts_tablet_manager.cc:1875] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:14.769784 7207 ts_tablet_manager.cc:1888] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:14.769846 7207 log.cc:1196] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/9ef6f9219f0d4f1884b948a07737a9f0
I20240502 01:59:14.770190 7207 ts_tablet_manager.cc:1909] T 9ef6f9219f0d4f1884b948a07737a9f0 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:14.770320 7066 ts_tablet_manager.cc:1888] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:14.770377 7066 log.cc:1196] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/9ef6f9219f0d4f1884b948a07737a9f0
I20240502 01:59:14.770584 7066 ts_tablet_manager.cc:1909] T 9ef6f9219f0d4f1884b948a07737a9f0 P 468dd27456c64d67b22e8a58af93d93d: Deleting consensus metadata
I20240502 01:59:14.771176 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 9ef6f9219f0d4f1884b948a07737a9f0 (table rebalancer_dynamic_table_new_name [id=dc57ac1188164725a3e7178ad7dbbd20]) successfully deleted
I20240502 01:59:14.771531 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 9ef6f9219f0d4f1884b948a07737a9f0 (table rebalancer_dynamic_table_new_name [id=dc57ac1188164725a3e7178ad7dbbd20]) successfully deleted
I20240502 01:59:14.771723 7208 ts_tablet_manager.cc:1888] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:14.771800 7208 log.cc:1196] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/9ef6f9219f0d4f1884b948a07737a9f0
I20240502 01:59:14.773190 7208 ts_tablet_manager.cc:1909] T 9ef6f9219f0d4f1884b948a07737a9f0 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:14.773866 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 9ef6f9219f0d4f1884b948a07737a9f0 (table rebalancer_dynamic_table_new_name [id=dc57ac1188164725a3e7178ad7dbbd20]) successfully deleted
I20240502 01:59:14.801653 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:14.806506 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:14.807293 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:14.808288 29451 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:14.811527 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:14.821110 26908 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 01:59:14.821372 26641 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:14.824115 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 894f4faba48f4e449a38f797e730ea4a (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.824396 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 894f4faba48f4e449a38f797e730ea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.824373 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 894f4faba48f4e449a38f797e730ea4a (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.824609 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 894f4faba48f4e449a38f797e730ea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.826542 26641 tablet_service.cc:1467] Processing CreateTablet for tablet afb8c2f7ea304bfaa32a7b12e74d8b4a (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.826787 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afb8c2f7ea304bfaa32a7b12e74d8b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.826831 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 894f4faba48f4e449a38f797e730ea4a (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.827052 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 894f4faba48f4e449a38f797e730ea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.830703 27043 tablet_service.cc:1467] Processing CreateTablet for tablet fdd5caaf71f240599843ca169abd14a4 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.830967 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd5caaf71f240599843ca169abd14a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.831709 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:14.835716 26909 tablet_service.cc:1467] Processing CreateTablet for tablet afb8c2f7ea304bfaa32a7b12e74d8b4a (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.835980 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afb8c2f7ea304bfaa32a7b12e74d8b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.835716 26908 tablet_service.cc:1467] Processing CreateTablet for tablet fdd5caaf71f240599843ca169abd14a4 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.838475 29195 tablet_service.cc:1467] Processing CreateTablet for tablet afb8c2f7ea304bfaa32a7b12e74d8b4a (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.838591 26908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd5caaf71f240599843ca169abd14a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.838570 29196 tablet_service.cc:1467] Processing CreateTablet for tablet fdd5caaf71f240599843ca169abd14a4 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=4145e60f85494be987285b1ddecdd5d6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:14.838721 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afb8c2f7ea304bfaa32a7b12e74d8b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.840780 26774 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:14.841359 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd5caaf71f240599843ca169abd14a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.848363 7226 tablet_bootstrap.cc:492] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:14.849246 7226 tablet_bootstrap.cc:654] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.849861 7223 tablet_bootstrap.cc:492] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:14.850697 7223 tablet_bootstrap.cc:654] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.851539 7225 tablet_bootstrap.cc:492] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:14.852418 7225 tablet_bootstrap.cc:654] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.868492 26508 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:14.868916 7228 tablet_bootstrap.cc:492] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:14.870549 7228 tablet_bootstrap.cc:654] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.883922 7222 tablet_bootstrap.cc:492] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:14.884217 7227 tablet_bootstrap.cc:492] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:14.885337 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 88179bbe6baf4e23818ceb3c29816827 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0]), partition=
I20240502 01:59:14.885577 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88179bbe6baf4e23818ceb3c29816827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.885721 7223 tablet_bootstrap.cc:492] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:14.885811 7223 ts_tablet_manager.cc:1367] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20240502 01:59:14.886284 7223 raft_consensus.cc:348] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:14.886397 7223 raft_consensus.cc:374] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.886423 7223 raft_consensus.cc:729] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.886538 7223 consensus_queue.cc:260] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:14.886801 7223 ts_tablet_manager.cc:1398] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.887392 27043 tablet_service.cc:1467] Processing CreateTablet for tablet 88179bbe6baf4e23818ceb3c29816827 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0]), partition=
I20240502 01:59:14.887596 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88179bbe6baf4e23818ceb3c29816827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.888240 7225 tablet_bootstrap.cc:492] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:14.888332 7225 ts_tablet_manager.cc:1367] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20240502 01:59:14.888571 26909 tablet_service.cc:1467] Processing CreateTablet for tablet 88179bbe6baf4e23818ceb3c29816827 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0]), partition=
I20240502 01:59:14.888792 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88179bbe6baf4e23818ceb3c29816827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:14.888747 7225 raft_consensus.cc:348] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.888841 7225 raft_consensus.cc:374] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.888867 7225 raft_consensus.cc:729] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.888971 7225 consensus_queue.cc:260] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.889215 7225 ts_tablet_manager.cc:1398] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:14.889376 7225 tablet_bootstrap.cc:492] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:14.890239 7225 tablet_bootstrap.cc:654] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.892817 7225 tablet_bootstrap.cc:492] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:14.892894 7225 ts_tablet_manager.cc:1367] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:14.893136 7223 tablet_bootstrap.cc:492] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:14.893285 7225 raft_consensus.cc:348] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:14.893373 7225 raft_consensus.cc:374] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.893396 7225 raft_consensus.cc:729] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.893491 7225 consensus_queue.cc:260] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:14.893735 7225 ts_tablet_manager.cc:1398] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.893960 7223 tablet_bootstrap.cc:654] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.898056 7227 tablet_bootstrap.cc:654] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.898795 7222 tablet_bootstrap.cc:654] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.899381 7227 tablet_bootstrap.cc:492] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:14.899417 7228 tablet_bootstrap.cc:492] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:14.899514 7227 ts_tablet_manager.cc:1367] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:59:14.899514 7228 ts_tablet_manager.cc:1367] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20240502 01:59:14.899925 7227 raft_consensus.cc:348] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.900015 7227 raft_consensus.cc:374] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.900043 7227 raft_consensus.cc:729] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.900092 7228 raft_consensus.cc:348] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.900184 7228 raft_consensus.cc:374] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.900147 7227 consensus_queue.cc:260] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.900213 7228 raft_consensus.cc:729] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.900315 7228 consensus_queue.cc:260] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.900426 7227 ts_tablet_manager.cc:1398] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:14.900573 7227 tablet_bootstrap.cc:492] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:14.900578 7228 ts_tablet_manager.cc:1398] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.900727 7228 tablet_bootstrap.cc:492] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:14.901430 7227 tablet_bootstrap.cc:654] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.901443 7223 tablet_bootstrap.cc:492] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:14.901516 7223 ts_tablet_manager.cc:1367] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 01:59:14.901564 7228 tablet_bootstrap.cc:654] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.901942 7223 raft_consensus.cc:348] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.902016 7223 raft_consensus.cc:374] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.902066 7223 raft_consensus.cc:729] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.902174 7223 consensus_queue.cc:260] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.902475 7223 ts_tablet_manager.cc:1398] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:14.904480 7227 tablet_bootstrap.cc:492] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:14.904556 7227 ts_tablet_manager.cc:1367] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 01:59:14.904966 7227 raft_consensus.cc:348] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.905092 7227 raft_consensus.cc:374] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.905143 7227 raft_consensus.cc:729] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.905287 7227 consensus_queue.cc:260] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.905643 7227 ts_tablet_manager.cc:1398] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:14.905815 7227 tablet_bootstrap.cc:492] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:14.906849 7227 tablet_bootstrap.cc:654] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.908061 7227 tablet_bootstrap.cc:492] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: No bootstrap required, opened a new log
I20240502 01:59:14.911657 7228 tablet_bootstrap.cc:492] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:14.911752 7228 ts_tablet_manager.cc:1367] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20240502 01:59:14.912604 7228 raft_consensus.cc:348] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.913368 7228 raft_consensus.cc:374] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.913436 7228 raft_consensus.cc:729] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.913605 7228 consensus_queue.cc:260] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.913925 7228 ts_tablet_manager.cc:1398] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:14.914213 7226 tablet_bootstrap.cc:492] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:14.914283 7227 ts_tablet_manager.cc:1367] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 01:59:14.914299 7226 ts_tablet_manager.cc:1367] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.000s
I20240502 01:59:14.914690 7227 raft_consensus.cc:348] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.914737 7226 raft_consensus.cc:348] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.914811 7227 raft_consensus.cc:374] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.914844 7226 raft_consensus.cc:374] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.914846 7227 raft_consensus.cc:729] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.914873 7226 raft_consensus.cc:729] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.914970 7227 consensus_queue.cc:260] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.914983 7226 consensus_queue.cc:260] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:14.915225 7227 ts_tablet_manager.cc:1398] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:14.915257 7226 ts_tablet_manager.cc:1398] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.915439 7226 tablet_bootstrap.cc:492] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:14.916247 7226 tablet_bootstrap.cc:654] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:14.916978 7222 tablet_bootstrap.cc:492] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:14.917042 7222 ts_tablet_manager.cc:1367] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 01:59:14.917482 7222 raft_consensus.cc:348] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:14.917585 7222 raft_consensus.cc:374] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.917605 7222 raft_consensus.cc:729] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.917696 7222 consensus_queue.cc:260] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:14.917971 7222 ts_tablet_manager.cc:1398] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:14.922166 7226 tablet_bootstrap.cc:492] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:14.922245 7226 ts_tablet_manager.cc:1367] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:59:14.922678 7226 raft_consensus.cc:348] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.924006 7226 raft_consensus.cc:374] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:14.924095 7226 raft_consensus.cc:729] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:14.924245 7226 consensus_queue.cc:260] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.924611 7226 ts_tablet_manager.cc:1398] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:14.926108 6828 raft_consensus.cc:482] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:14.926193 6828 raft_consensus.cc:504] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.926509 6828 leader_election.cc:288] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:14.926847 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88179bbe6baf4e23818ceb3c29816827" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:14.926954 27062 raft_consensus.cc:2454] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:14.927206 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88179bbe6baf4e23818ceb3c29816827" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:14.927309 26930 raft_consensus.cc:2454] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:14.927620 26730 leader_election.cc:302] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:14.927920 6828 raft_consensus.cc:2781] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:14.927973 6828 raft_consensus.cc:482] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:14.928014 6828 raft_consensus.cc:3037] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.928992 6828 raft_consensus.cc:504] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.929314 6828 leader_election.cc:288] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:14.929742 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88179bbe6baf4e23818ceb3c29816827" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:14.929838 26930 raft_consensus.cc:3037] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.929980 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88179bbe6baf4e23818ceb3c29816827" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:14.930088 27062 raft_consensus.cc:3037] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:14.930773 26930 raft_consensus.cc:2454] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:14.930933 27062 raft_consensus.cc:2454] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:14.931818 26730 leader_election.cc:302] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:14.932238 6953 raft_consensus.cc:2781] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:14.932329 6953 raft_consensus.cc:686] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:14.932454 6953 consensus_queue.cc:237] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:14.934734 26511 catalog_manager.cc:5484] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:14.951503 26659 consensus_queue.cc:237] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:14.952636 29488 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:14.952862 26930 raft_consensus.cc:1261] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:14.953135 6386 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:14.953660 6386 consensus_queue.cc:1035] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:14.954687 6865 raft_consensus.cc:2932] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:14.956105 29488 raft_consensus.cc:2932] T b546a99821624f0588a4c0f59a5ec5a3 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:14.957525 26511 catalog_manager.cc:5484] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 2 to 4, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New cstate: current_term: 2 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:14.957866 26930 raft_consensus.cc:2932] T b546a99821624f0588a4c0f59a5ec5a3 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:14.962486 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet b546a99821624f0588a4c0f59a5ec5a3 with delete_type TABLET_DATA_TOMBSTONED (TS 68e5b1eeb04448fea818a14991eab87c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:14.963315 7208 tablet_replica.cc:330] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:14.963478 7208 raft_consensus.cc:2229] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:14.963590 7208 raft_consensus.cc:2258] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:14.964000 7208 ts_tablet_manager.cc:1875] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:14.965947 7208 ts_tablet_manager.cc:1888] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240502 01:59:14.965996 7208 log.cc:1196] T b546a99821624f0588a4c0f59a5ec5a3 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/b546a99821624f0588a4c0f59a5ec5a3
I20240502 01:59:14.966490 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet b546a99821624f0588a4c0f59a5ec5a3 (table rebalancer_extra_table_44 [id=2911511be60e49369b8d77b5f55380f0]) successfully deleted
I20240502 01:59:15.044512 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:15.050822 26927 raft_consensus.cc:1261] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:15.051040 27062 raft_consensus.cc:1261] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:15.051398 6953 consensus_queue.cc:1035] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.051573 6828 consensus_queue.cc:1035] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.055795 26511 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet 88179bbe6baf4e23818ceb3c29816827 (table rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0]). Expected version 1 got 0
I20240502 01:59:15.056887 7081 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:15.057190 7081 tablet.cc:1620] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:15.057358 7079 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:15.057616 7079 tablet.cc:1620] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:15.057719 7337 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:15.057991 7337 tablet.cc:1620] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:15.060957 26507 catalog_manager.cc:5576] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379) does not have the latest schema for tablet 88179bbe6baf4e23818ceb3c29816827 (table rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0]). Expected version 1 got 0
I20240502 01:59:15.062832 26511 catalog_manager.cc:5576] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635) does not have the latest schema for tablet 88179bbe6baf4e23818ceb3c29816827 (table rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0]). Expected version 1 got 0
I20240502 01:59:15.064054 6536 raft_consensus.cc:482] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.064136 6536 raft_consensus.cc:504] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.064476 6536 leader_election.cc:288] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:15.065008 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd5caaf71f240599843ca169abd14a4" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:15.065131 29216 raft_consensus.cc:2454] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:15.065651 26861 leader_election.cc:302] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:15.065953 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd5caaf71f240599843ca169abd14a4" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:15.066080 27062 raft_consensus.cc:2454] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 0.
I20240502 01:59:15.066802 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0] alter complete (version 1)
I20240502 01:59:15.067556 6536 raft_consensus.cc:2781] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.067623 6536 raft_consensus.cc:482] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.067660 6536 raft_consensus.cc:3037] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.068454 6536 raft_consensus.cc:504] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.068811 6536 leader_election.cc:288] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:15.069569 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd5caaf71f240599843ca169abd14a4" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:15.069568 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd5caaf71f240599843ca169abd14a4" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:15.069679 27062 raft_consensus.cc:3037] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.069679 29216 raft_consensus.cc:3037] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.070600 27062 raft_consensus.cc:2454] T fdd5caaf71f240599843ca169abd14a4 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:15.070601 29216 raft_consensus.cc:2454] T fdd5caaf71f240599843ca169abd14a4 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:15.071250 26861 leader_election.cc:302] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:15.071507 6536 raft_consensus.cc:2781] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.071597 6536 raft_consensus.cc:686] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:15.071722 6536 consensus_queue.cc:237] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.074069 26511 catalog_manager.cc:5484] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 0 to 1, leader changed from <none> to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:15.171500 26510 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 01:59:15.176327 7081 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:15.176641 7081 tablet.cc:1620] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: MemRowSet was empty: no flush needed.
I20240502 01:59:15.177973 7079 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:15.178303 7079 tablet.cc:1620] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: MemRowSet was empty: no flush needed.
I20240502 01:59:15.179263 7337 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 01:59:15.179603 7337 tablet.cc:1620] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: MemRowSet was empty: no flush needed.
I20240502 01:59:15.183817 26513 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=d21919420b7441a29dcf1d3c85a7bdb0] alter complete (version 2)
I20240502 01:59:15.215466 6386 raft_consensus.cc:482] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.215571 6386 raft_consensus.cc:504] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.216003 6386 leader_election.cc:288] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:15.216284 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb8c2f7ea304bfaa32a7b12e74d8b4a" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:15.216372 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb8c2f7ea304bfaa32a7b12e74d8b4a" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:15.216415 26927 raft_consensus.cc:2454] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:15.216513 29216 raft_consensus.cc:2454] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:15.216732 26595 leader_election.cc:302] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:15.216977 6386 raft_consensus.cc:2781] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.217046 6386 raft_consensus.cc:482] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.217087 6386 raft_consensus.cc:3037] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.217958 6386 raft_consensus.cc:504] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.218262 6386 leader_election.cc:288] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:15.218544 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb8c2f7ea304bfaa32a7b12e74d8b4a" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:15.218582 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb8c2f7ea304bfaa32a7b12e74d8b4a" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:15.218655 26927 raft_consensus.cc:3037] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.218694 29216 raft_consensus.cc:3037] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.219524 26927 raft_consensus.cc:2454] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:15.219631 29216 raft_consensus.cc:2454] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:15.219780 26595 leader_election.cc:302] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:15.219923 6386 raft_consensus.cc:2781] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.220005 6386 raft_consensus.cc:686] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:15.220132 6386 consensus_queue.cc:237] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.222575 26511 catalog_manager.cc:5484] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:15.287642 26511 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 01:59:15.300513 7081 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:15.301812 7079 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:15.303342 7337 tablet.cc:1722] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 01:59:15.307788 6828 raft_consensus.cc:482] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.307884 6828 raft_consensus.cc:504] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:15.308238 6828 leader_election.cc:288] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:15.308516 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f4faba48f4e449a38f797e730ea4a" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:15.308530 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f4faba48f4e449a38f797e730ea4a" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:15.308646 26659 raft_consensus.cc:2454] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:15.308643 29350 raft_consensus.cc:2454] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 0.
I20240502 01:59:15.308921 26729 leader_election.cc:302] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:15.309075 6828 raft_consensus.cc:2781] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.309140 6828 raft_consensus.cc:482] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.309188 6828 raft_consensus.cc:3037] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.309881 6828 raft_consensus.cc:504] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:15.310216 6828 leader_election.cc:288] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Requested vote from peers 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:15.310391 26511 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table_new_name [id=d21919420b7441a29dcf1d3c85a7bdb0] alter complete (version 3)
I20240502 01:59:15.311287 26659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f4faba48f4e449a38f797e730ea4a" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:15.311398 26659 raft_consensus.cc:3037] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.312291 26659 raft_consensus.cc:2454] T 894f4faba48f4e449a38f797e730ea4a P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:15.312538 26729 leader_election.cc:302] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:15.312692 6828 raft_consensus.cc:2781] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.312780 6828 raft_consensus.cc:686] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:15.312909 6828 consensus_queue.cc:237] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:15.313004 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f4faba48f4e449a38f797e730ea4a" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:15.313098 29350 raft_consensus.cc:3037] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.313990 29350 raft_consensus.cc:2454] T 894f4faba48f4e449a38f797e730ea4a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:15.316422 26510 catalog_manager.cc:5484] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 0 to 1, leader changed from <none> to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:15.340238 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:15.346736 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:15.347244 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:15.347605 29451 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:15.355348 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:15.367733 26640 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:15.384181 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:15.388865 26774 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:15.411691 26510 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:15.411849 26510 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 01:59:15.414045 26510 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: Sending DeleteTablet for 3 replicas of tablet 88179bbe6baf4e23818ceb3c29816827
I20240502 01:59:15.414580 26774 tablet_service.cc:1514] Processing DeleteTablet for tablet 88179bbe6baf4e23818ceb3c29816827 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:15 UTC) from {username='slave'} at 127.0.0.1:44132
I20240502 01:59:15.414901 27043 tablet_service.cc:1514] Processing DeleteTablet for tablet 88179bbe6baf4e23818ceb3c29816827 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:15 UTC) from {username='slave'} at 127.0.0.1:37980
I20240502 01:59:15.414940 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet 88179bbe6baf4e23818ceb3c29816827 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 01:59:15 UTC) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:15.415087 7208 tablet_replica.cc:330] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: stopping tablet replica
I20240502 01:59:15.415237 7208 raft_consensus.cc:2229] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:15.415362 7208 raft_consensus.cc:2258] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:15.416244 7208 ts_tablet_manager.cc:1875] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:15.418510 7208 ts_tablet_manager.cc:1888] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:15.418582 7208 log.cc:1196] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/wals/88179bbe6baf4e23818ceb3c29816827
I20240502 01:59:15.418828 7208 ts_tablet_manager.cc:1909] T 88179bbe6baf4e23818ceb3c29816827 P 68e5b1eeb04448fea818a14991eab87c: Deleting consensus metadata
I20240502 01:59:15.419405 26502 catalog_manager.cc:4830] TS 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): tablet 88179bbe6baf4e23818ceb3c29816827 (table rebalancer_dynamic_table_new_name [id=d21919420b7441a29dcf1d3c85a7bdb0]) successfully deleted
I20240502 01:59:15.421388 7360 tablet_replica.cc:330] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:15.421568 7360 raft_consensus.cc:2229] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:15.421694 7360 raft_consensus.cc:2258] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:15.422132 7359 tablet_replica.cc:330] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: stopping tablet replica
I20240502 01:59:15.422322 7359 raft_consensus.cc:2229] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Raft consensus shutting down.
I20240502 01:59:15.422514 7360 ts_tablet_manager.cc:1875] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:15.422555 7359 raft_consensus.cc:2258] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:15.423311 7359 ts_tablet_manager.cc:1875] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 01:59:15.425220 7359 ts_tablet_manager.cc:1888] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:15.425290 7359 log.cc:1196] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/wals/88179bbe6baf4e23818ceb3c29816827
I20240502 01:59:15.425295 7360 ts_tablet_manager.cc:1888] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 01:59:15.425354 7360 log.cc:1196] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/88179bbe6baf4e23818ceb3c29816827
I20240502 01:59:15.425678 7360 ts_tablet_manager.cc:1909] T 88179bbe6baf4e23818ceb3c29816827 P 842c1b011b5d4c77bfd9b9591111502e: Deleting consensus metadata
I20240502 01:59:15.426512 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet 88179bbe6baf4e23818ceb3c29816827 (table rebalancer_dynamic_table_new_name [id=d21919420b7441a29dcf1d3c85a7bdb0]) successfully deleted
I20240502 01:59:15.433182 7359 ts_tablet_manager.cc:1909] T 88179bbe6baf4e23818ceb3c29816827 P b290c1bf893444078ed721f25995d124: Deleting consensus metadata
I20240502 01:59:15.433960 26501 catalog_manager.cc:4830] TS b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): tablet 88179bbe6baf4e23818ceb3c29816827 (table rebalancer_dynamic_table_new_name [id=d21919420b7441a29dcf1d3c85a7bdb0]) successfully deleted
I20240502 01:59:15.445302 26661 consensus_queue.cc:237] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:15.446624 29350 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:15.446882 27062 raft_consensus.cc:1261] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer 998b57e2badc4d779c586c6579149114: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:15.447041 6865 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:15.447238 6386 consensus_queue.cc:1035] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:15.448872 7379 raft_consensus.cc:2932] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:15.449229 27062 raft_consensus.cc:2932] T 8209cefcc49042b18fdd97d845d167e2 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:15.449321 29350 raft_consensus.cc:2932] T 8209cefcc49042b18fdd97d845d167e2 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:15.452246 26510 catalog_manager.cc:5484] T 8209cefcc49042b18fdd97d845d167e2 P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 2 to 4, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) evicted. New cstate: current_term: 2 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:15.455585 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet 8209cefcc49042b18fdd97d845d167e2 with delete_type TABLET_DATA_TOMBSTONED (TS 7c693d0933c64d9babc07f404df45cd1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35930
I20240502 01:59:15.455885 7385 tablet_replica.cc:330] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: stopping tablet replica
I20240502 01:59:15.456069 7385 raft_consensus.cc:2229] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:15.456233 7385 raft_consensus.cc:2258] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:15.456663 7385 ts_tablet_manager.cc:1875] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:15.459321 7385 ts_tablet_manager.cc:1888] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240502 01:59:15.459386 7385 log.cc:1196] T 8209cefcc49042b18fdd97d845d167e2 P 7c693d0933c64d9babc07f404df45cd1: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/wals/8209cefcc49042b18fdd97d845d167e2
I20240502 01:59:15.459908 26502 catalog_manager.cc:4830] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): tablet 8209cefcc49042b18fdd97d845d167e2 (table rebalancer_extra_table_40 [id=227548e4a0b54a26bf8780cc896a7e07]) successfully deleted
I20240502 01:59:15.516153 26514 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 01:59:15.523792 29451 tablet_service.cc:1467] Processing CreateTablet for tablet b22135977bb840ed958a414b31489016 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5]), partition=
I20240502 01:59:15.524041 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22135977bb840ed958a414b31489016. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.526409 7423 tablet_bootstrap.cc:492] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:15.527384 7423 tablet_bootstrap.cc:654] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.527420 29328 tablet_service.cc:1467] Processing CreateTablet for tablet b22135977bb840ed958a414b31489016 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5]), partition=
I20240502 01:59:15.527478 29058 tablet_service.cc:1467] Processing CreateTablet for tablet b22135977bb840ed958a414b31489016 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5]), partition=
I20240502 01:59:15.527673 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22135977bb840ed958a414b31489016. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.527699 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22135977bb840ed958a414b31489016. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.528540 7423 tablet_bootstrap.cc:492] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:15.528671 7423 ts_tablet_manager.cc:1367] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:15.529505 7423 raft_consensus.cc:348] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.529693 7425 tablet_bootstrap.cc:492] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:15.529698 7423 raft_consensus.cc:374] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.529743 7423 raft_consensus.cc:729] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.531129 7423 consensus_queue.cc:260] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.532352 7423 ts_tablet_manager.cc:1398] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:15.532636 7426 tablet_bootstrap.cc:492] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:15.533154 7425 tablet_bootstrap.cc:654] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.533324 7426 tablet_bootstrap.cc:654] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.535485 7425 tablet_bootstrap.cc:492] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:15.535566 7425 ts_tablet_manager.cc:1367] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:15.536007 7425 raft_consensus.cc:348] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.536101 7425 raft_consensus.cc:374] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.536132 7425 raft_consensus.cc:729] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.536234 7425 consensus_queue.cc:260] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.536511 7425 ts_tablet_manager.cc:1398] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:15.540345 7426 tablet_bootstrap.cc:492] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:15.540436 7426 ts_tablet_manager.cc:1367] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 01:59:15.543081 7426 raft_consensus.cc:348] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.543255 7426 raft_consensus.cc:374] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.543344 7426 raft_consensus.cc:729] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.543512 7426 consensus_queue.cc:260] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.551324 7426 ts_tablet_manager.cc:1398] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20240502 01:59:15.573508 6536 consensus_queue.cc:1035] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.595610 6872 consensus_queue.cc:1035] T fdd5caaf71f240599843ca169abd14a4 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.609901 29328 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 01:59:15.610566 29451 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:15.612330 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:15.622895 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:15.630877 26640 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:15.639330 26774 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:15.644647 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:15.645285 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:15.647589 6678 raft_consensus.cc:482] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.647680 6678 raft_consensus.cc:504] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.648015 6678 leader_election.cc:288] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:15.649256 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22135977bb840ed958a414b31489016" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1" is_pre_election: true
I20240502 01:59:15.649394 29489 raft_consensus.cc:2454] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:15.650139 29286 leader_election.cc:302] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:15.650933 7025 raft_consensus.cc:2781] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.650996 7025 raft_consensus.cc:482] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.651050 7025 raft_consensus.cc:3037] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.652020 7025 raft_consensus.cc:504] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.652341 7025 leader_election.cc:288] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:15.652976 7379 consensus_queue.cc:1035] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.654619 29489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22135977bb840ed958a414b31489016" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c693d0933c64d9babc07f404df45cd1"
I20240502 01:59:15.654632 7379 consensus_queue.cc:1035] T afb8c2f7ea304bfaa32a7b12e74d8b4a P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.654731 29489 raft_consensus.cc:3037] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.655699 29489 raft_consensus.cc:2454] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:15.655992 29286 leader_election.cc:302] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1; no voters:
I20240502 01:59:15.657243 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22135977bb840ed958a414b31489016" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:15.657369 29079 raft_consensus.cc:2454] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:15.657544 6678 raft_consensus.cc:2781] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.657707 6678 raft_consensus.cc:686] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:15.657838 6678 consensus_queue.cc:237] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:15.662004 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22135977bb840ed958a414b31489016" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:15.662147 29080 raft_consensus.cc:3037] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.663157 29080 raft_consensus.cc:2454] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:15.668432 26512 catalog_manager.cc:5484] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 0 to 1, leader changed from <none> to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:15.714351 6953 consensus_queue.cc:1035] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.718786 6828 consensus_queue.cc:1035] T 894f4faba48f4e449a38f797e730ea4a P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.723428 29215 consensus_queue.cc:237] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:15.724524 26794 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:15.724526 29489 raft_consensus.cc:1261] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:15.724871 7115 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:15.725044 7221 consensus_queue.cc:1035] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:15.726773 7221 raft_consensus.cc:2932] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:15.727510 26794 raft_consensus.cc:2932] T 273b6d65e3df42ffb072e2129e832a92 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:15.729238 29489 raft_consensus.cc:2932] T 273b6d65e3df42ffb072e2129e832a92 P 7c693d0933c64d9babc07f404df45cd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:15.730175 26512 catalog_manager.cc:5484] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index 2 to 4, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) evicted. New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:15.735085 29328 tablet_service.cc:1514] Processing DeleteTablet for tablet 273b6d65e3df42ffb072e2129e832a92 with delete_type TABLET_DATA_TOMBSTONED (TS 468dd27456c64d67b22e8a58af93d93d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34760
I20240502 01:59:15.735481 7476 tablet_replica.cc:330] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: stopping tablet replica
I20240502 01:59:15.735733 7476 raft_consensus.cc:2229] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:15.735900 7476 raft_consensus.cc:2258] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:15.736393 7476 ts_tablet_manager.cc:1875] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:15.739042 7476 ts_tablet_manager.cc:1888] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240502 01:59:15.739107 7476 log.cc:1196] T 273b6d65e3df42ffb072e2129e832a92 P 468dd27456c64d67b22e8a58af93d93d: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/wals/273b6d65e3df42ffb072e2129e832a92
I20240502 01:59:15.739696 26499 catalog_manager.cc:4830] TS 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): tablet 273b6d65e3df42ffb072e2129e832a92 (table rebalancer_extra_table_48 [id=09549d32a04340f6aa7540e255c610c5]) successfully deleted
I20240502 01:59:15.800788 26514 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46396:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 01:59:15.805501 29489 raft_consensus.cc:1261] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:15.805613 29080 raft_consensus.cc:1261] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:15.806138 7446 consensus_queue.cc:1035] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.807030 7446 consensus_queue.cc:1035] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:15.810163 26510 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet b22135977bb840ed958a414b31489016 (table rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5]). Expected version 1 got 0
I20240502 01:59:15.819054 26510 catalog_manager.cc:5576] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601) does not have the latest schema for tablet b22135977bb840ed958a414b31489016 (table rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5]). Expected version 1 got 0
I20240502 01:59:15.821513 26506 catalog_manager.cc:5576] TS 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259) does not have the latest schema for tablet b22135977bb840ed958a414b31489016 (table rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5]). Expected version 1 got 0
I20240502 01:59:15.825219 7518 tablet.cc:1722] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:15.825559 7518 tablet.cc:1620] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d: MemRowSet was empty: no flush needed.
I20240502 01:59:15.827777 26509 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46396:
name: "rebalancer_extra_table_74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:15.831416 7520 tablet.cc:1722] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:15.831708 7520 tablet.cc:1620] T b22135977bb840ed958a414b31489016 P f3d18a019013487ab801599cfb80e4b4: MemRowSet was empty: no flush needed.
I20240502 01:59:15.831982 7519 tablet.cc:1722] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 01:59:15.833420 7519 tablet.cc:1620] T b22135977bb840ed958a414b31489016 P 7c693d0933c64d9babc07f404df45cd1: MemRowSet was empty: no flush needed.
I20240502 01:59:15.836665 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc69c007af34db18b67f7d6c4f20d80 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.836936 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc69c007af34db18b67f7d6c4f20d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.837463 26640 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc69c007af34db18b67f7d6c4f20d80 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.837512 26509 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 35b243e102134ca6be77b68ec0929e02: rebalancer_dynamic_table [id=6dbbd31ec24245ed82cea52c1b60e1c5] alter complete (version 1)
I20240502 01:59:15.837710 26640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc69c007af34db18b67f7d6c4f20d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.838013 26641 tablet_service.cc:1467] Processing CreateTablet for tablet 01cdd2a4d12d452c8af39d4ff6f71108 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.837335 29196 tablet_service.cc:1467] Processing CreateTablet for tablet d659e57b5ed94e30b3a821c896e974a0 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.838255 26641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cdd2a4d12d452c8af39d4ff6f71108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.838397 29196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d659e57b5ed94e30b3a821c896e974a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.840257 27043 tablet_service.cc:1467] Processing CreateTablet for tablet d659e57b5ed94e30b3a821c896e974a0 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.840389 7423 tablet_bootstrap.cc:492] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1: Bootstrap starting.
I20240502 01:59:15.840533 27043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d659e57b5ed94e30b3a821c896e974a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.841259 7423 tablet_bootstrap.cc:654] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.843063 26774 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc69c007af34db18b67f7d6c4f20d80 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.843183 26773 tablet_service.cc:1467] Processing CreateTablet for tablet 01cdd2a4d12d452c8af39d4ff6f71108 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.843449 26774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc69c007af34db18b67f7d6c4f20d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.844034 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cdd2a4d12d452c8af39d4ff6f71108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.844244 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 01cdd2a4d12d452c8af39d4ff6f71108 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.844484 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cdd2a4d12d452c8af39d4ff6f71108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.846237 29058 tablet_service.cc:1467] Processing CreateTablet for tablet d659e57b5ed94e30b3a821c896e974a0 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=ab9a36a37aba4bd3ac568559d8a78688]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:15.846860 7425 tablet_bootstrap.cc:492] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
I20240502 01:59:15.847790 7425 tablet_bootstrap.cc:654] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.848091 7521 tablet_bootstrap.cc:492] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:15.848448 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d659e57b5ed94e30b3a821c896e974a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:15.849171 7525 tablet_bootstrap.cc:492] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:15.850262 7522 tablet_bootstrap.cc:492] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Bootstrap starting.
I20240502 01:59:15.850870 7524 tablet_bootstrap.cc:492] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:15.851286 7426 tablet_bootstrap.cc:492] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4: Bootstrap starting.
I20240502 01:59:15.851784 7524 tablet_bootstrap.cc:654] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.852197 7426 tablet_bootstrap.cc:654] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.853152 7521 tablet_bootstrap.cc:654] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.854396 7522 tablet_bootstrap.cc:654] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.855217 7525 tablet_bootstrap.cc:654] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.855695 7423 tablet_bootstrap.cc:492] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1: No bootstrap required, opened a new log
I20240502 01:59:15.855773 7423 ts_tablet_manager.cc:1367] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 01:59:15.856220 7423 raft_consensus.cc:348] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.856318 7423 raft_consensus.cc:374] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.856346 7423 raft_consensus.cc:729] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.856456 7423 consensus_queue.cc:260] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.856739 7423 ts_tablet_manager.cc:1398] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:15.861742 7524 tablet_bootstrap.cc:492] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:15.861871 7524 ts_tablet_manager.cc:1367] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 01:59:15.862416 7524 raft_consensus.cc:348] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.863255 7426 tablet_bootstrap.cc:492] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4: No bootstrap required, opened a new log
I20240502 01:59:15.863327 7426 ts_tablet_manager.cc:1367] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 01:59:15.863327 7425 tablet_bootstrap.cc:492] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d: No bootstrap required, opened a new log
I20240502 01:59:15.863411 7425 ts_tablet_manager.cc:1367] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 01:59:15.863731 7426 raft_consensus.cc:348] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.863842 7426 raft_consensus.cc:374] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.863869 7426 raft_consensus.cc:729] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.863853 7425 raft_consensus.cc:348] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.863957 7425 raft_consensus.cc:374] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.863986 7425 raft_consensus.cc:729] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.863966 7426 consensus_queue.cc:260] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.864087 7425 consensus_queue.cc:260] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.864226 7426 ts_tablet_manager.cc:1398] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:15.864354 7425 ts_tablet_manager.cc:1398] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:15.864688 7524 raft_consensus.cc:374] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.864737 7524 raft_consensus.cc:729] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.864857 7524 consensus_queue.cc:260] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.867627 7524 ts_tablet_manager.cc:1398] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:15.867786 7524 tablet_bootstrap.cc:492] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:15.868764 7521 tablet_bootstrap.cc:492] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:15.868856 7521 ts_tablet_manager.cc:1367] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 01:59:15.869431 7521 raft_consensus.cc:348] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.870236 7521 raft_consensus.cc:374] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.870278 7521 raft_consensus.cc:729] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.870405 7521 consensus_queue.cc:260] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.870748 7521 ts_tablet_manager.cc:1398] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:15.870947 7521 tablet_bootstrap.cc:492] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:15.872056 7521 tablet_bootstrap.cc:654] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.875020 7524 tablet_bootstrap.cc:654] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:15.880414 7522 tablet_bootstrap.cc:492] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: No bootstrap required, opened a new log
I20240502 01:59:15.881196 7525 tablet_bootstrap.cc:492] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c: No bootstrap required, opened a new log
I20240502 01:59:15.881378 7525 ts_tablet_manager.cc:1367] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 01:59:15.881520 7522 ts_tablet_manager.cc:1367] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20240502 01:59:15.882683 7522 raft_consensus.cc:348] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.882915 7525 raft_consensus.cc:348] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.883111 7525 raft_consensus.cc:374] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.883213 7522 raft_consensus.cc:374] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.883296 7525 raft_consensus.cc:729] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.883550 7525 consensus_queue.cc:260] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.883381 7522 raft_consensus.cc:729] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.883888 7522 consensus_queue.cc:260] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.886329 7525 ts_tablet_manager.cc:1398] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:59:15.887871 7522 ts_tablet_manager.cc:1398] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 01:59:15.892328 7524 tablet_bootstrap.cc:492] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124: No bootstrap required, opened a new log
I20240502 01:59:15.892426 7524 ts_tablet_manager.cc:1367] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 01:59:15.892871 7521 tablet_bootstrap.cc:492] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114: No bootstrap required, opened a new log
I20240502 01:59:15.893383 7524 raft_consensus.cc:348] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.893594 7521 ts_tablet_manager.cc:1367] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 01:59:15.894068 7521 raft_consensus.cc:348] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.894239 7524 raft_consensus.cc:374] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.894327 7524 raft_consensus.cc:729] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.894515 7524 consensus_queue.cc:260] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.894661 7521 raft_consensus.cc:374] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:15.894986 7524 ts_tablet_manager.cc:1398] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:15.895211 7521 raft_consensus.cc:729] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:15.895427 7521 consensus_queue.cc:260] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.895937 7521 ts_tablet_manager.cc:1398] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:15.914677 7429 raft_consensus.cc:482] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.914780 7429 raft_consensus.cc:504] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.915149 7429 leader_election.cc:288] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:15.916060 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc69c007af34db18b67f7d6c4f20d80" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:15.916184 26794 raft_consensus.cc:2454] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:15.916445 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc69c007af34db18b67f7d6c4f20d80" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:15.916577 26661 raft_consensus.cc:2454] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 0.
I20240502 01:59:15.916882 29421 leader_election.cc:302] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:15.917116 7221 raft_consensus.cc:482] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.917212 7221 raft_consensus.cc:504] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.917264 5771 raft_consensus.cc:2781] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.917310 5771 raft_consensus.cc:482] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.917343 5771 raft_consensus.cc:3037] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.917579 7221 leader_election.cc:288] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:15.918409 5771 raft_consensus.cc:504] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.918740 5771 leader_election.cc:288] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:15.920075 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d659e57b5ed94e30b3a821c896e974a0" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:15.920190 27062 raft_consensus.cc:2454] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:15.921006 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d659e57b5ed94e30b3a821c896e974a0" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:15.921139 29080 raft_consensus.cc:2454] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 0.
I20240502 01:59:15.921519 29150 leader_election.cc:302] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:15.922382 29196 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:15.924224 7221 raft_consensus.cc:2781] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.924284 7221 raft_consensus.cc:482] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.924321 7221 raft_consensus.cc:3037] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.925302 7221 raft_consensus.cc:504] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.925637 7221 leader_election.cc:288] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:15.926520 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d659e57b5ed94e30b3a821c896e974a0" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:15.926628 27064 raft_consensus.cc:3037] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.927615 27064 raft_consensus.cc:2454] T d659e57b5ed94e30b3a821c896e974a0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:15.927826 7447 raft_consensus.cc:482] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.927907 7447 raft_consensus.cc:504] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.928218 7447 leader_election.cc:288] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:15.928647 6865 raft_consensus.cc:482] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:15.928726 6865 raft_consensus.cc:504] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.929045 6865 leader_election.cc:288] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:15.929450 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cdd2a4d12d452c8af39d4ff6f71108" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:15.929561 29348 raft_consensus.cc:2454] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:15.929844 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cdd2a4d12d452c8af39d4ff6f71108" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:15.929953 26794 raft_consensus.cc:2454] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 0.
I20240502 01:59:15.930280 26592 leader_election.cc:302] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:15.931232 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cdd2a4d12d452c8af39d4ff6f71108" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:15.931380 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cdd2a4d12d452c8af39d4ff6f71108" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:15.931468 26794 raft_consensus.cc:2454] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 0.
I20240502 01:59:15.931258 7379 raft_consensus.cc:2781] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:15.931699 7379 raft_consensus.cc:482] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:15.931761 7379 raft_consensus.cc:3037] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.932731 7379 raft_consensus.cc:504] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.933028 7379 leader_election.cc:288] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Requested vote from peers b290c1bf893444078ed721f25995d124 (127.19.43.194:38007), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:15.933979 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cdd2a4d12d452c8af39d4ff6f71108" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:15.934096 29348 raft_consensus.cc:3037] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.935004 29348 raft_consensus.cc:2454] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:15.937255 26793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cdd2a4d12d452c8af39d4ff6f71108" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:15.937348 26793 raft_consensus.cc:3037] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.938221 26793 raft_consensus.cc:2454] T 01cdd2a4d12d452c8af39d4ff6f71108 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998b57e2badc4d779c586c6579149114 in term 1.
I20240502 01:59:15.938510 26594 leader_election.cc:302] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:15.938665 7379 raft_consensus.cc:2781] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.938751 7379 raft_consensus.cc:686] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [term 1 LEADER]: Becoming Leader. State: Replica: 998b57e2badc4d779c586c6579149114, State: Running, Role: LEADER
I20240502 01:59:15.938866 7379 consensus_queue.cc:237] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:15.939074 29150 leader_election.cc:302] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:15.939236 7115 raft_consensus.cc:2781] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.939327 7115 raft_consensus.cc:686] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:15.939432 7115 consensus_queue.cc:237] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:15.939587 29285 leader_election.cc:302] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:15.939937 7446 raft_consensus.cc:2741] T 01cdd2a4d12d452c8af39d4ff6f71108 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20240502 01:59:15.940732 29079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d659e57b5ed94e30b3a821c896e974a0" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:15.940836 29079 raft_consensus.cc:3037] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.941764 29079 raft_consensus.cc:2454] T d659e57b5ed94e30b3a821c896e974a0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:15.942719 26661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc69c007af34db18b67f7d6c4f20d80" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:15.942818 26661 raft_consensus.cc:3037] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.942835 26508 catalog_manager.cc:5484] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:15.943730 26661 raft_consensus.cc:2454] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:15.943873 26510 catalog_manager.cc:5484] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998b57e2badc4d779c586c6579149114 (127.19.43.193). New cstate: current_term: 1 leader_uuid: "998b57e2badc4d779c586c6579149114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:15.944016 29421 leader_election.cc:302] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c693d0933c64d9babc07f404df45cd1, 998b57e2badc4d779c586c6579149114; no voters:
I20240502 01:59:15.944504 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc69c007af34db18b67f7d6c4f20d80" candidate_uuid: "7c693d0933c64d9babc07f404df45cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:15.944593 26794 raft_consensus.cc:3037] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:15.945547 26794 raft_consensus.cc:2454] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c693d0933c64d9babc07f404df45cd1 in term 1.
I20240502 01:59:15.947238 7550 raft_consensus.cc:2781] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:15.947315 7550 raft_consensus.cc:686] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c693d0933c64d9babc07f404df45cd1, State: Running, Role: LEADER
I20240502 01:59:15.948132 7550 consensus_queue.cc:237] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:15.949865 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:15.957507 29451 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:15.958779 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:15.963461 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:15.973003 26508 catalog_manager.cc:5484] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200). New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:15.982389 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:15.984872 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:15.991362 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:16.052670 27064 consensus_queue.cc:237] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:16.053817 29348 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 01:59:16.054181 6980 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.054458 29215 raft_consensus.cc:1261] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer 68e5b1eeb04448fea818a14991eab87c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 01:59:16.054808 6980 consensus_queue.cc:1035] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.055877 4345 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:16.056689 29348 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:16.056905 29215 raft_consensus.cc:2932] T 8afd4c5362464c4cba43d5ef28b50c29 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:16.060492 26508 catalog_manager.cc:5484] T 8afd4c5362464c4cba43d5ef28b50c29 P 68e5b1eeb04448fea818a14991eab87c reported cstate change: config changed from index 4 to 6, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New cstate: current_term: 2 leader_uuid: "68e5b1eeb04448fea818a14991eab87c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:16.063414 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 8afd4c5362464c4cba43d5ef28b50c29 with delete_type TABLET_DATA_TOMBSTONED (TS f3d18a019013487ab801599cfb80e4b4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:16.065559 7574 tablet_replica.cc:330] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:16.065714 7574 raft_consensus.cc:2229] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:16.065807 7574 raft_consensus.cc:2258] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:16.066267 7574 ts_tablet_manager.cc:1875] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:16.068828 7574 ts_tablet_manager.cc:1888] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 01:59:16.068972 7574 log.cc:1196] T 8afd4c5362464c4cba43d5ef28b50c29 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/8afd4c5362464c4cba43d5ef28b50c29
I20240502 01:59:16.069746 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 8afd4c5362464c4cba43d5ef28b50c29 (table rebalancer_extra_table_6 [id=e47d800688104d6e9f6e21c739dc1355]) successfully deleted
I20240502 01:59:16.202718 29196 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:16.207409 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:16.208985 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:16.211012 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:16.218763 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:16.222512 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:16.223070 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:16.247704 29451 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:16.291966 29348 consensus_queue.cc:237] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:16.293140 26927 raft_consensus.cc:1261] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:16.293183 26658 raft_consensus.cc:1261] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:16.293553 7447 consensus_queue.cc:1035] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:16.293758 7446 consensus_queue.cc:1035] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:16.295464 7446 raft_consensus.cc:2932] T 7f701a3437f84a84b6e3290266d24c94 P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.295848 26930 raft_consensus.cc:2932] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.299687 26511 catalog_manager.cc:5484] T 7f701a3437f84a84b6e3290266d24c94 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.302918 29058 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f701a3437f84a84b6e3290266d24c94 with delete_type TABLET_DATA_TOMBSTONED (TS f3d18a019013487ab801599cfb80e4b4 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:45946
I20240502 01:59:16.303136 7574 tablet_replica.cc:330] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: stopping tablet replica
I20240502 01:59:16.303293 7574 raft_consensus.cc:2229] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:16.303407 7574 raft_consensus.cc:2258] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:16.303840 7574 ts_tablet_manager.cc:1875] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:16.306052 7574 ts_tablet_manager.cc:1888] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:16.306128 7574 log.cc:1196] T 7f701a3437f84a84b6e3290266d24c94 P f3d18a019013487ab801599cfb80e4b4: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/wals/7f701a3437f84a84b6e3290266d24c94
I20240502 01:59:16.306630 26499 catalog_manager.cc:4830] TS f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): tablet 7f701a3437f84a84b6e3290266d24c94 (table rebalancer_extra_table_24 [id=39530783f9744d63b3514731b2c8c2b8]) successfully deleted
I20240502 01:59:16.311352 26658 raft_consensus.cc:2932] T 7f701a3437f84a84b6e3290266d24c94 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.346205 26794 consensus_queue.cc:237] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:16.347442 26658 raft_consensus.cc:1261] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:16.347728 29216 raft_consensus.cc:1261] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:16.347810 7684 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.348105 7336 consensus_queue.cc:1035] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 01:59:16.348418 26730 consensus_peers.cc:485] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc99f8bfb06c4bcda6111b218125826c. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.350009 7336 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.350416 26658 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.352291 29216 raft_consensus.cc:2932] T dc99f8bfb06c4bcda6111b218125826c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.354148 26509 catalog_manager.cc:5484] T dc99f8bfb06c4bcda6111b218125826c P 998b57e2badc4d779c586c6579149114 reported cstate change: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.375480 29216 consensus_queue.cc:237] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:16.376874 29488 raft_consensus.cc:1261] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.376938 29149 consensus_peers.cc:485] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 398742b2cb3e41f1bc16fa37a5168809. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.377293 7115 consensus_queue.cc:1035] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.382364 29079 raft_consensus.cc:1261] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.382710 7549 consensus_queue.cc:1035] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.384711 7549 raft_consensus.cc:2932] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.387281 29488 raft_consensus.cc:2932] T 398742b2cb3e41f1bc16fa37a5168809 P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.387424 29079 raft_consensus.cc:2932] T 398742b2cb3e41f1bc16fa37a5168809 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.388526 26513 catalog_manager.cc:5484] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.408979 6386 consensus_queue.cc:1035] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:16.412380 6865 consensus_queue.cc:1035] T 01cdd2a4d12d452c8af39d4ff6f71108 P 998b57e2badc4d779c586c6579149114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:16.421437 26794 consensus_queue.cc:237] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
W20240502 01:59:16.422744 26727 consensus_peers.cc:485] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 -> Peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303): Couldn't send request to peer 468dd27456c64d67b22e8a58af93d93d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b265eec55ee1406b9ff99c159304fa65. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.423020 29216 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:16.423048 26930 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:16.423350 7684 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.423502 7687 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.425361 7687 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:16.425781 26930 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:16.427634 29215 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:16.429765 26512 catalog_manager.cc:5484] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 5 to 6, VOTER 468dd27456c64d67b22e8a58af93d93d (127.19.43.199) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:16.435922 7551 consensus_queue.cc:1035] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:16.439850 7550 consensus_queue.cc:1035] T 8fc69c007af34db18b67f7d6c4f20d80 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:16.453404 26930 consensus_queue.cc:237] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:16.454684 29215 raft_consensus.cc:1261] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.454912 29348 raft_consensus.cc:1261] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.455066 26861 consensus_peers.cc:485] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02e988cd07184e97921e7376da20f251. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.455180 7752 consensus_queue.cc:1035] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.455363 7751 consensus_queue.cc:1035] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.456977 7751 raft_consensus.cc:2932] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.457383 29348 raft_consensus.cc:2932] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.458935 29215 raft_consensus.cc:2932] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.460963 7707 consensus_queue.cc:1035] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:16.461174 26511 catalog_manager.cc:5484] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.464286 7221 consensus_queue.cc:1035] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:16.485230 26794 consensus_queue.cc:237] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:16.486449 29079 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 01:59:16.486797 7684 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.486897 27064 raft_consensus.cc:1261] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 01:59:16.486959 26730 consensus_peers.cc:485] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 565aab51035c4f328e49eca53eddf621. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.487246 7684 consensus_queue.cc:1035] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:16.488914 7336 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.489339 27064 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.490785 29079 raft_consensus.cc:2932] T 565aab51035c4f328e49eca53eddf621 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.491925 26511 catalog_manager.cc:5484] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index 5 to 6, VOTER 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:16.513600 29488 consensus_queue.cc:237] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:16.514935 29348 raft_consensus.cc:1261] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.515079 29419 consensus_peers.cc:485] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 -> Peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Couldn't send request to peer f3d18a019013487ab801599cfb80e4b4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e66b19b5551b4011bf1bbb6234a2aba3. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.515455 27064 raft_consensus.cc:1261] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Refusing update from remote peer 7c693d0933c64d9babc07f404df45cd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.515496 7551 consensus_queue.cc:1035] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.515802 7551 consensus_queue.cc:1035] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.517833 7551 raft_consensus.cc:2932] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:16.518273 27064 raft_consensus.cc:2932] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:16.522117 26509 catalog_manager.cc:5484] T e66b19b5551b4011bf1bbb6234a2aba3 P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New cstate: current_term: 1 leader_uuid: "7c693d0933c64d9babc07f404df45cd1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:16.522336 29348 raft_consensus.cc:2932] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f3d18a019013487ab801599cfb80e4b4 (127.19.43.197) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:16.539793 29215 consensus_queue.cc:237] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:16.541000 29488 raft_consensus.cc:1261] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.541018 29348 raft_consensus.cc:1261] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.541118 29150 consensus_peers.cc:485] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11a922c5a8994e92a4112d89ce8a9f3a. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.541328 7221 consensus_queue.cc:1035] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.541702 7707 consensus_queue.cc:1035] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.543502 7221 raft_consensus.cc:2932] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.545063 29488 raft_consensus.cc:2932] T 11a922c5a8994e92a4112d89ce8a9f3a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.545872 29348 raft_consensus.cc:2932] T 11a922c5a8994e92a4112d89ce8a9f3a P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.546674 26512 catalog_manager.cc:5484] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.567404 26794 consensus_queue.cc:237] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:16.568701 29348 raft_consensus.cc:1261] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.568795 29215 raft_consensus.cc:1261] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.569094 7684 consensus_queue.cc:1035] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:16.569269 26729 consensus_peers.cc:485] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e96f6477ca7a497c82c01ef0b2d2747f. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.569360 6828 consensus_queue.cc:1035] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.571097 7684 raft_consensus.cc:2932] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.571498 29215 raft_consensus.cc:2932] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.574193 29348 raft_consensus.cc:2932] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.574503 26509 catalog_manager.cc:5484] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 2, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:16.596357 29079 consensus_queue.cc:237] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:16.597635 29488 raft_consensus.cc:1261] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.597754 29215 raft_consensus.cc:1261] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.598007 29015 consensus_peers.cc:485] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 -> Peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007): Couldn't send request to peer b290c1bf893444078ed721f25995d124. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4caef235274947e48765e1c11669112a. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.598143 7555 consensus_queue.cc:1035] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.598316 5443 consensus_queue.cc:1035] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.600075 5443 raft_consensus.cc:2932] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.600466 29488 raft_consensus.cc:2932] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.603199 26509 catalog_manager.cc:5484] T 4caef235274947e48765e1c11669112a P 7c693d0933c64d9babc07f404df45cd1 reported cstate change: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.603974 29215 raft_consensus.cc:2932] T 4caef235274947e48765e1c11669112a P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b290c1bf893444078ed721f25995d124 (127.19.43.194) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:16.625770 29348 consensus_queue.cc:237] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:16.627303 26930 raft_consensus.cc:1261] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.627466 29286 consensus_peers.cc:485] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12420fbcddc5446b857d2e6a01f40aaa. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.627655 29079 raft_consensus.cc:1261] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.627709 7651 consensus_queue.cc:1035] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.628027 7651 consensus_queue.cc:1035] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.630218 7446 raft_consensus.cc:2932] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:16.630556 29079 raft_consensus.cc:2932] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:16.632560 26930 raft_consensus.cc:2932] T 12420fbcddc5446b857d2e6a01f40aaa P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:16.634965 26513 catalog_manager.cc:5484] T 12420fbcddc5446b857d2e6a01f40aaa P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:16.651935 29215 consensus_queue.cc:237] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:16.653196 29348 raft_consensus.cc:1261] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:16.653553 29150 consensus_peers.cc:485] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): Couldn't send request to peer 842c1b011b5d4c77bfd9b9591111502e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68f002fcea6f4378954808dbfb5f10e6. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:16.653590 7221 consensus_queue.cc:1035] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.653752 26794 raft_consensus.cc:1261] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:16.654433 7221 consensus_queue.cc:1035] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:16.656145 7707 raft_consensus.cc:2932] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.656558 26794 raft_consensus.cc:2932] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.657840 29350 raft_consensus.cc:2932] T 68f002fcea6f4378954808dbfb5f10e6 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.660059 26512 catalog_manager.cc:5484] T 68f002fcea6f4378954808dbfb5f10e6 P b290c1bf893444078ed721f25995d124 reported cstate change: config changed from index -1 to 2, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) added. New cstate: current_term: 1 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:16.762555 7937 ts_tablet_manager.cc:902] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:16.763044 7937 tablet_copy_client.cc:308] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.198:40103
I20240502 01:59:16.763480 29221 tablet_copy_service.cc:140] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Received BeginTabletCopySession request for tablet 398742b2cb3e41f1bc16fa37a5168809 from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:43975)
I20240502 01:59:16.763573 29221 tablet_copy_service.cc:161] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Beginning new tablet copy session on tablet 398742b2cb3e41f1bc16fa37a5168809 from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:43975: session id = b290c1bf893444078ed721f25995d124-398742b2cb3e41f1bc16fa37a5168809
I20240502 01:59:16.764359 29221 tablet_copy_source_session.cc:205] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:16.765143 7937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 398742b2cb3e41f1bc16fa37a5168809. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:16.767345 7937 tablet_copy_client.cc:783] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:16.767554 7937 tablet_copy_client.cc:647] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:16.768592 7937 tablet_copy_client.cc:523] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:16.769639 7937 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:16.771241 7942 ts_tablet_manager.cc:902] T dc99f8bfb06c4bcda6111b218125826c P 7c693d0933c64d9babc07f404df45cd1: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
F20240502 01:59:16.771422 7942 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-052566382186437e8e0245de9f60c30c.queue_time_us, tablet-download-pool-052566382186437e8e0245de9f60c30c.queue_time_us) (tablet-download-pool-052566382186437e8e0245de9f60c30c.run_cpu_time_us, tablet-download-pool-052566382186437e8e0245de9f60c30c.run_cpu_time_us) (tablet-download-pool-052566382186437e8e0245de9f60c30c.run_wall_time_us, tablet-download-pool-052566382186437e8e0245de9f60c30c.run_wall_time_us) (tablet-download-pool-213c37d37f1c4a3c8fbc48deafa9edac.queue_time_us, tablet-download-pool-213c37d37f1c4a3c8fbc48deafa9edac.queue_time_us) (tablet-download-pool-213c37d37f1c4a3c8fbc48deafa9edac.run_cpu_time_us, tablet-download-pool-213c37d37f1c4a3c8fbc48deafa9edac.run_cpu_time_us) (tablet-download-pool-213c37d37f1c4a3c8fbc48deafa9edac.run_wall_time_us, tablet-download-pool-213c37d37f1c4a3c8fbc48deafa9edac.run_wall_time_us) (tablet-download-pool-2c150446ded240a2946d5afa10ecd1df.queue_time_us, tablet-download-pool-2c150446ded240a2946d5afa10ecd1df.queue_time_us) (tablet-download-pool-2c150446ded240a2946d5afa10ecd1df.run_cpu_time_us, tablet-download-pool-2c150446ded240a2946d5afa10ecd1df.run_cpu_time_us) (tablet-download-pool-2c150446ded240a2946d5afa10ecd1df.run_wall_time_us, tablet-download-pool-2c150446ded240a2946d5afa10ecd1df.run_wall_time_us) (tablet-download-pool-49ac3a00934944bdb5c9e440045532a3.queue_time_us, tablet-download-pool-49ac3a00934944bdb5c9e440045532a3.queue_time_us) (tablet-download-pool-49ac3a00934944bdb5c9e440045532a3.run_cpu_time_us, tablet-download-pool-49ac3a00934944bdb5c9e440045532a3.run_cpu_time_us) (tablet-download-pool-49ac3a00934944bdb5c9e440045532a3.run_wall_time_us, tablet-download-pool-49ac3a00934944bdb5c9e440045532a3.run_wall_time_us) (tablet-download-pool-4a528d9bbfa2488e8ce1e7b99b9e4d49.queue_time_us, tablet-download-pool-4a528d9bbfa2488e8ce1e7b99b9e4d49.queue_time_us) (tablet-download-pool-4a528d9bbfa2488e8ce1e7b99b9e4d49.run_cpu_time_us, tablet-download-pool-4a528d9bbfa2488e8ce1e7b99b9e4d49.run_cpu_time_us) (tablet-download-pool-4a528d9bbfa2488e8ce1e7b99b9e4d49.run_wall_time_us, tablet-download-pool-4a528d9bbfa2488e8ce1e7b99b9e4d49.run_wall_time_us) (tablet-download-pool-626774246b9445bc838427358e08f2cd.queue_time_us, tablet-download-pool-626774246b9445bc838427358e08f2cd.queue_time_us) (tablet-download-pool-626774246b9445bc838427358e08f2cd.run_cpu_time_us, tablet-download-pool-626774246b9445bc838427358e08f2cd.run_cpu_time_us) (tablet-download-pool-626774246b9445bc838427358e08f2cd.run_wall_time_us, tablet-download-pool-626774246b9445bc838427358e08f2cd.run_wall_time_us) (tablet-download-pool-6e2ebe00a52240cb9ac5fc8fea9d7151.queue_time_us, tablet-download-pool-6e2ebe00a52240cb9ac5fc8fea9d7151.queue_time_us) (tablet-download-pool-6e2ebe00a52240cb9ac5fc8fea9d7151.run_cpu_time_us, tablet-download-pool-6e2ebe00a52240cb9ac5fc8fea9d7151.run_cpu_time_us) (tablet-download-pool-6e2ebe00a52240cb9ac5fc8fea9d7151.run_wall_time_us, tablet-download-pool-6e2ebe00a52240cb9ac5fc8fea9d7151.run_wall_time_us) (tablet-download-pool-6e9b0960a2a94c3fb90902c126ccd034.queue_time_us, tablet-download-pool-6e9b0960a2a94c3fb90902c126ccd034.queue_time_us) (tablet-download-pool-6e9b0960a2a94c3fb90902c126ccd034.run_cpu_time_us, tablet-download-pool-6e9b0960a2a94c3fb90902c126ccd034.run_cpu_time_us) (tablet-download-pool-6e9b0960a2a94c3fb90902c126ccd034.run_wall_time_us, tablet-download-pool-6e9b0960a2a94c3fb90902c126ccd034.run_wall_time_us) (tablet-download-pool-7b2aaf8eaa844c0390909ccfc0b4bb09.queue_time_us, tablet-download-pool-7b2aaf8eaa844c0390909ccfc0b4bb09.queue_time_us) (tablet-download-pool-7b2aaf8eaa844c0390909ccfc0b4bb09.run_cpu_time_us, tablet-download-pool-7b2aaf8eaa844c0390909ccfc0b4bb09.run_cpu_time_us) (tablet-download-pool-7b2aaf8eaa844c0390909ccfc0b4bb09.run_wall_time_us, tablet-download-pool-7b2aaf8eaa844c0390909ccfc0b4bb09.run_wall_time_us) (tablet-download-pool-b5439ff11ca04925bf46e736a5a53487.queue_time_us, tablet-download-pool-b5439ff11ca04925bf46e736a5a53487.queue_time_us) (tablet-download-pool-b5439ff11ca04925bf46e736a5a53487.run_cpu_time_us, tablet-download-pool-b5439ff11ca04925bf46e736a5a53487.run_cpu_time_us) (tablet-download-pool-b5439ff11ca04925bf46e736a5a53487.run_wall_time_us, tablet-download-pool-b5439ff11ca04925bf46e736a5a53487.run_wall_time_us) (tablet-download-pool-b546a99821624f0588a4c0f59a5ec5a3.queue_time_us, tablet-download-pool-b546a99821624f0588a4c0f59a5ec5a3.queue_time_us) (tablet-download-pool-b546a99821624f0588a4c0f59a5ec5a3.run_cpu_time_us, tablet-download-pool-b546a99821624f0588a4c0f59a5ec5a3.run_cpu_time_us) (tablet-download-pool-b546a99821624f0588a4c0f59a5ec5a3.run_wall_time_us, tablet-download-pool-b546a99821624f0588a4c0f59a5ec5a3.run_wall_time_us) (tablet-download-pool-c46a7ee366464769b8498c4208ce8e9b.queue_time_us, tablet-download-pool-c46a7ee366464769b8498c4208ce8e9b.queue_time_us) (tablet-download-pool-c46a7ee366464769b8498c4208ce8e9b.run_cpu_time_us, tablet-download-pool-c46a7ee366464769b8498c4208ce8e9b.run_cpu_time_us) (tablet-download-pool-c46a7ee366464769b8498c4208ce8e9b.run_wall_time_us, tablet-download-pool-c46a7ee366464769b8498c4208ce8e9b.run_wall_time_us) (tablet-download-pool-dc99f8bfb06c4bcda6111b218125826c.queue_time_us, tablet-download-pool-dc99f8bfb06c4bcda6111b218125826c.queue_time_us) (tablet-download-pool-dfdd6d487b57479cb48a92bc770947ce.queue_time_us, tablet-download-pool-dfdd6d487b57479cb48a92bc770947ce.queue_time_us) (tablet-download-pool-dfdd6d487b57479cb48a92bc770947ce.run_cpu_time_us, tablet-download-pool-dfdd6d487b57479cb48a92bc770947ce.run_cpu_time_us) (tablet-download-pool-dfdd6d487b57479cb48a92bc770947ce.run_wall_time_us, tablet-download-pool-dfdd6d487b57479cb48a92bc770947ce.run_wall_time_us) (tablet-download-pool-e1cd4681de8645ff92b479ccdd3eac37.queue_time_us, tablet-download-pool-e1cd4681de8645ff92b479ccdd3eac37.queue_time_us) (tablet-download-pool-e1cd4681de8645ff92b479ccdd3eac37.run_cpu_time_us, tablet-download-pool-e1cd4681de8645ff92b479ccdd3eac37.run_cpu_time_us) (tablet-download-pool-e1cd4681de8645ff92b479ccdd3eac37.run_wall_time_us, tablet-download-pool-e1cd4681de8645ff92b479ccdd3eac37.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f99e0b13dcd google::LogMessage::Fail() at ??:0
@ 0x7f99e0b17b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f99e0b137cc google::LogMessage::Flush() at ??:0
@ 0x7f99e0b14f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20240502 01:59:16.794083 7937 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:16.794234 29196 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:16.794348 7937 tablet_bootstrap.cc:492] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:59:16.794498 7937 ts_tablet_manager.cc:1367] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20240502 01:59:16.794940 7937 raft_consensus.cc:348] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:16.795049 7937 raft_consensus.cc:729] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:16.795195 7937 consensus_queue.cc:260] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:16.795469 7937 ts_tablet_manager.cc:1398] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:16.795796 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:16.796574 29221 tablet_copy_service.cc:342] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Request end of tablet copy session b290c1bf893444078ed721f25995d124-398742b2cb3e41f1bc16fa37a5168809 received from {username='slave'} at 127.19.43.194:43975
I20240502 01:59:16.796638 29221 tablet_copy_service.cc:433] P 0d83fdd1e6e04cf5a5602994d1dbc0df: ending tablet copy session b290c1bf893444078ed721f25995d124-398742b2cb3e41f1bc16fa37a5168809 on tablet 398742b2cb3e41f1bc16fa37a5168809 with peer b290c1bf893444078ed721f25995d124
@ 0x7f99e1f7d044 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f99e1f61f98 kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f99e1f5fefc kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f99e4b1277a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f99e4b1ba45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
I20240502 01:59:16.820670 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:16.822393 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
@ 0x7f99e4b9c76f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f99e4bb04f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
I20240502 01:59:16.828826 7958 ts_tablet_manager.cc:902] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:16.829857 7958 tablet_copy_client.cc:308] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
@ 0x7f99e4b9b226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
I20240502 01:59:16.830286 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet b265eec55ee1406b9ff99c159304fa65 from peer 468dd27456c64d67b22e8a58af93d93d ({username='slave'} at 127.19.43.199:33039)
I20240502 01:59:16.830370 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet b265eec55ee1406b9ff99c159304fa65 from peer 468dd27456c64d67b22e8a58af93d93d at {username='slave'} at 127.19.43.199:33039: session id = 468dd27456c64d67b22e8a58af93d93d-b265eec55ee1406b9ff99c159304fa65
I20240502 01:59:16.831241 26815 tablet_copy_source_session.cc:205] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:16.832072 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b265eec55ee1406b9ff99c159304fa65. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f99e4baac0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x55a10c7a775c std::function<>::operator()() at ??:0
@ 0x7f99e1f64e06 kudu::ThreadPool::DispatchThread() at ??:0
I20240502 01:59:16.837769 7958 tablet_copy_client.cc:783] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:16.837903 7958 tablet_copy_client.cc:647] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Starting download of 1 WAL segments...
@ 0x7f99e1f656ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
I20240502 01:59:16.839663 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:16.841063 7958 tablet_copy_client.cc:523] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
@ 0x7f99e1f6703b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x55a10c7a775c std::function<>::operator()() at ??:0
I20240502 01:59:16.842566 7958 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap starting.
@ 0x7f99e1f54807 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f99e27926db start_thread at ??:0
@ 0x7f99dfcca71f clone at ??:0
I20240502 01:59:16.857683 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:16.874477 7958 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:16.874727 7958 tablet_bootstrap.cc:492] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: Bootstrap complete.
I20240502 01:59:16.874874 7958 ts_tablet_manager.cc:1367] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.008s
I20240502 01:59:16.875211 7958 raft_consensus.cc:348] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:16.875308 7958 raft_consensus.cc:729] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Initialized, Role: FOLLOWER
I20240502 01:59:16.875412 7958 consensus_queue.cc:260] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:16.875687 7958 ts_tablet_manager.cc:1398] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:16.877063 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 468dd27456c64d67b22e8a58af93d93d-b265eec55ee1406b9ff99c159304fa65 received from {username='slave'} at 127.19.43.199:33039
I20240502 01:59:16.877131 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 468dd27456c64d67b22e8a58af93d93d-b265eec55ee1406b9ff99c159304fa65 on tablet b265eec55ee1406b9ff99c159304fa65 with peer 468dd27456c64d67b22e8a58af93d93d
I20240502 01:59:16.903681 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:16.992779 7981 ts_tablet_manager.cc:902] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: Initiating tablet copy from peer b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:16.993317 7981 tablet_copy_client.cc:308] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.194:38007
I20240502 01:59:16.993736 26815 tablet_copy_service.cc:140] P b290c1bf893444078ed721f25995d124: Received BeginTabletCopySession request for tablet e96f6477ca7a497c82c01ef0b2d2747f from peer 998b57e2badc4d779c586c6579149114 ({username='slave'} at 127.19.43.193:37351)
I20240502 01:59:16.993834 26815 tablet_copy_service.cc:161] P b290c1bf893444078ed721f25995d124: Beginning new tablet copy session on tablet e96f6477ca7a497c82c01ef0b2d2747f from peer 998b57e2badc4d779c586c6579149114 at {username='slave'} at 127.19.43.193:37351: session id = 998b57e2badc4d779c586c6579149114-e96f6477ca7a497c82c01ef0b2d2747f
I20240502 01:59:16.994809 26815 tablet_copy_source_session.cc:205] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:16.995728 7981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96f6477ca7a497c82c01ef0b2d2747f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:16.998263 7981 tablet_copy_client.cc:783] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:16.998400 7981 tablet_copy_client.cc:647] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:16.999882 7981 tablet_copy_client.cc:523] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:17.001044 7981 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:17.004351 7981 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:17.004624 7981 tablet_bootstrap.cc:492] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: Bootstrap complete.
I20240502 01:59:17.004837 7981 ts_tablet_manager.cc:1367] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20240502 01:59:17.005275 7981 raft_consensus.cc:348] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.005391 7981 raft_consensus.cc:729] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:17.005504 7981 consensus_queue.cc:260] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.005800 7981 ts_tablet_manager.cc:1398] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:17.006251 26815 tablet_copy_service.cc:342] P b290c1bf893444078ed721f25995d124: Request end of tablet copy session 998b57e2badc4d779c586c6579149114-e96f6477ca7a497c82c01ef0b2d2747f received from {username='slave'} at 127.19.43.193:37351
I20240502 01:59:17.006332 26815 tablet_copy_service.cc:433] P b290c1bf893444078ed721f25995d124: ending tablet copy session 998b57e2badc4d779c586c6579149114-e96f6477ca7a497c82c01ef0b2d2747f on tablet e96f6477ca7a497c82c01ef0b2d2747f with peer 998b57e2badc4d779c586c6579149114
I20240502 01:59:17.006664 7988 ts_tablet_manager.cc:902] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:17.007246 7988 tablet_copy_client.cc:308] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.198:40103
I20240502 01:59:17.008059 29221 tablet_copy_service.cc:140] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Received BeginTabletCopySession request for tablet 11a922c5a8994e92a4112d89ce8a9f3a from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:36349)
I20240502 01:59:17.008158 29221 tablet_copy_service.cc:161] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Beginning new tablet copy session on tablet 11a922c5a8994e92a4112d89ce8a9f3a from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:36349: session id = 842c1b011b5d4c77bfd9b9591111502e-11a922c5a8994e92a4112d89ce8a9f3a
I20240502 01:59:17.008894 29221 tablet_copy_source_session.cc:205] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:17.009680 7988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a922c5a8994e92a4112d89ce8a9f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:17.011890 7937 ts_tablet_manager.cc:902] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:17.012292 7937 tablet_copy_client.cc:308] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:59:17.012998 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet 4caef235274947e48765e1c11669112a from peer b290c1bf893444078ed721f25995d124 ({username='slave'} at 127.19.43.194:41057)
I20240502 01:59:17.013099 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet 4caef235274947e48765e1c11669112a from peer b290c1bf893444078ed721f25995d124 at {username='slave'} at 127.19.43.194:41057: session id = b290c1bf893444078ed721f25995d124-4caef235274947e48765e1c11669112a
I20240502 01:59:17.013177 7988 tablet_copy_client.cc:783] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:17.013314 7988 tablet_copy_client.cc:647] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:17.014084 29102 tablet_copy_source_session.cc:205] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:17.014834 7988 tablet_copy_client.cc:523] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:17.014874 7937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4caef235274947e48765e1c11669112a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:17.016584 7988 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:17.016975 7937 tablet_copy_client.cc:783] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:17.017120 7937 tablet_copy_client.cc:647] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:17.019829 7937 tablet_copy_client.cc:523] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:17.021034 7937 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: Bootstrap starting.
I20240502 01:59:17.022472 7988 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:17.022709 7988 tablet_bootstrap.cc:492] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:17.022922 7988 ts_tablet_manager.cc:1367] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 01:59:17.023485 7988 raft_consensus.cc:348] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:17.023641 7988 raft_consensus.cc:729] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:17.023845 7988 consensus_queue.cc:260] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:17.024447 7937 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:17.024657 7937 tablet_bootstrap.cc:492] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: Bootstrap complete.
I20240502 01:59:17.024787 7937 ts_tablet_manager.cc:1367] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:17.025204 7937 raft_consensus.cc:348] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:17.025297 7937 raft_consensus.cc:729] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b290c1bf893444078ed721f25995d124, State: Initialized, Role: FOLLOWER
I20240502 01:59:17.025425 7937 consensus_queue.cc:260] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:17.025775 7937 ts_tablet_manager.cc:1398] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:59:17.025802 7988 ts_tablet_manager.cc:1398] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:17.026276 29221 tablet_copy_service.cc:342] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-11a922c5a8994e92a4112d89ce8a9f3a received from {username='slave'} at 127.19.43.195:36349
I20240502 01:59:17.026355 29221 tablet_copy_service.cc:433] P 0d83fdd1e6e04cf5a5602994d1dbc0df: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-11a922c5a8994e92a4112d89ce8a9f3a on tablet 11a922c5a8994e92a4112d89ce8a9f3a with peer 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:17.027505 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session b290c1bf893444078ed721f25995d124-4caef235274947e48765e1c11669112a received from {username='slave'} at 127.19.43.194:41057
I20240502 01:59:17.027583 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session b290c1bf893444078ed721f25995d124-4caef235274947e48765e1c11669112a on tablet 4caef235274947e48765e1c11669112a with peer b290c1bf893444078ed721f25995d124
I20240502 01:59:17.061758 7981 ts_tablet_manager.cc:902] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: Initiating tablet copy from peer 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:17.062211 7981 tablet_copy_client.cc:308] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.195:40379
I20240502 01:59:17.062645 26950 tablet_copy_service.cc:140] P 842c1b011b5d4c77bfd9b9591111502e: Received BeginTabletCopySession request for tablet 02e988cd07184e97921e7376da20f251 from peer 998b57e2badc4d779c586c6579149114 ({username='slave'} at 127.19.43.193:40669)
I20240502 01:59:17.062734 26950 tablet_copy_service.cc:161] P 842c1b011b5d4c77bfd9b9591111502e: Beginning new tablet copy session on tablet 02e988cd07184e97921e7376da20f251 from peer 998b57e2badc4d779c586c6579149114 at {username='slave'} at 127.19.43.193:40669: session id = 998b57e2badc4d779c586c6579149114-02e988cd07184e97921e7376da20f251
I20240502 01:59:17.063450 26950 tablet_copy_source_session.cc:205] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:17.064173 7981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e988cd07184e97921e7376da20f251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:17.066331 7981 tablet_copy_client.cc:783] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:17.066533 7981 tablet_copy_client.cc:647] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:17.067629 7981 tablet_copy_client.cc:523] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:17.069162 7981 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
I20240502 01:59:17.072686 7981 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:17.072945 7981 tablet_bootstrap.cc:492] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: Bootstrap complete.
I20240502 01:59:17.073107 7981 ts_tablet_manager.cc:1367] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:17.073472 7981 raft_consensus.cc:348] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.073582 7981 raft_consensus.cc:729] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:17.073717 7981 consensus_queue.cc:260] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.074465 7981 ts_tablet_manager.cc:1398] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:59:17.074889 26950 tablet_copy_service.cc:342] P 842c1b011b5d4c77bfd9b9591111502e: Request end of tablet copy session 998b57e2badc4d779c586c6579149114-02e988cd07184e97921e7376da20f251 received from {username='slave'} at 127.19.43.193:40669
I20240502 01:59:17.074965 26950 tablet_copy_service.cc:433] P 842c1b011b5d4c77bfd9b9591111502e: ending tablet copy session 998b57e2badc4d779c586c6579149114-02e988cd07184e97921e7376da20f251 on tablet 02e988cd07184e97921e7376da20f251 with peer 998b57e2badc4d779c586c6579149114
I20240502 01:59:17.113413 7988 ts_tablet_manager.cc:902] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: Initiating tablet copy from peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:17.113839 7988 tablet_copy_client.cc:308] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.198:40103
I20240502 01:59:17.114313 29221 tablet_copy_service.cc:140] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Received BeginTabletCopySession request for tablet 68f002fcea6f4378954808dbfb5f10e6 from peer 842c1b011b5d4c77bfd9b9591111502e ({username='slave'} at 127.19.43.195:36349)
I20240502 01:59:17.114449 29221 tablet_copy_service.cc:161] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Beginning new tablet copy session on tablet 68f002fcea6f4378954808dbfb5f10e6 from peer 842c1b011b5d4c77bfd9b9591111502e at {username='slave'} at 127.19.43.195:36349: session id = 842c1b011b5d4c77bfd9b9591111502e-68f002fcea6f4378954808dbfb5f10e6
I20240502 01:59:17.115432 29221 tablet_copy_source_session.cc:205] T 68f002fcea6f4378954808dbfb5f10e6 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:17.116192 7988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68f002fcea6f4378954808dbfb5f10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:17.118434 7988 tablet_copy_client.cc:783] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:17.118561 7988 tablet_copy_client.cc:647] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:17.119585 7988 tablet_copy_client.cc:523] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:17.121218 7988 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap starting.
I20240502 01:59:17.124132 7988 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:17.124361 7988 tablet_bootstrap.cc:492] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: Bootstrap complete.
I20240502 01:59:17.124509 7988 ts_tablet_manager.cc:1367] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:59:17.124866 7988 raft_consensus.cc:348] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:17.124955 7988 raft_consensus.cc:729] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Initialized, Role: FOLLOWER
I20240502 01:59:17.125053 7988 consensus_queue.cc:260] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
W20240502 01:59:17.125694 29016 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.125803 29286 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.125825 29016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.125921 29286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.126322 29150 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.126427 29150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 01:59:17.126482 7988 ts_tablet_manager.cc:1398] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20240502 01:59:17.126492 26730 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.126590 26730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.126607 26595 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.126713 26595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 01:59:17.126894 29221 tablet_copy_service.cc:342] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Request end of tablet copy session 842c1b011b5d4c77bfd9b9591111502e-68f002fcea6f4378954808dbfb5f10e6 received from {username='slave'} at 127.19.43.195:36349
I20240502 01:59:17.126950 29221 tablet_copy_service.cc:433] P 0d83fdd1e6e04cf5a5602994d1dbc0df: ending tablet copy session 842c1b011b5d4c77bfd9b9591111502e-68f002fcea6f4378954808dbfb5f10e6 on tablet 68f002fcea6f4378954808dbfb5f10e6 with peer 842c1b011b5d4c77bfd9b9591111502e
W20240502 01:59:17.128294 26998 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.128398 26998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.128763 26862 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.128854 26862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.130345 8018 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 01:59:17.127364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.43.200:36259 (local address 127.19.43.197:38249)
0502 01:59:17.127596 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0502 01:59:17.127865 (+ 269us) client_negotiation.cc:174] Beginning negotiation
0502 01:59:17.127963 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 01:59:17.129478 (+ 1515us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
W20240502 01:59:17.130553 29016 consensus_peers.cc:485] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.130617 29016 consensus_peers.cc:485] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.130656 29016 consensus_peers.cc:485] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.130693 29016 consensus_peers.cc:485] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.131541 26862 consensus_peers.cc:485] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132040 29150 consensus_peers.cc:485] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132112 29150 consensus_peers.cc:485] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132158 29150 consensus_peers.cc:485] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132202 29150 consensus_peers.cc:485] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132381 8017 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 01:59:17.127234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.43.200:36259 (local address 127.19.43.194:40005)
0502 01:59:17.127963 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0502 01:59:17.127977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0502 01:59:17.129004 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 01:59:17.131582 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":59,"threads_started":1}
W20240502 01:59:17.132444 26862 consensus_peers.cc:485] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132500 26862 consensus_peers.cc:485] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132509 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.132542 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.132539 26862 consensus_peers.cc:485] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132571 26730 consensus_peers.cc:485] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132608 26730 consensus_peers.cc:485] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.132645 26730 consensus_peers.cc:485] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.133003 29286 consensus_peers.cc:485] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.133077 29286 consensus_peers.cc:485] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.133108 29286 consensus_peers.cc:485] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.136479 26998 consensus_peers.cc:485] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.136548 26998 consensus_peers.cc:485] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.136890 26595 consensus_peers.cc:485] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.136998 26595 consensus_peers.cc:485] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.137914 26595 consensus_peers.cc:485] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.155191 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:17.155968 29286 consensus_peers.cc:485] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.166705 26730 consensus_peers.cc:485] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.178480 29016 consensus_peers.cc:485] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.181999 26730 consensus_peers.cc:485] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.186242 29286 consensus_peers.cc:485] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.194110 26595 consensus_peers.cc:485] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.196384 26998 consensus_peers.cc:485] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.213711 26595 consensus_peers.cc:485] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.215147 26998 consensus_peers.cc:485] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.215664 29286 consensus_peers.cc:485] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.219734 26730 consensus_peers.cc:485] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:17.241196 8069 ts_tablet_manager.cc:902] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: Initiating tablet copy from peer 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:17.241744 8069 tablet_copy_client.cc:308] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.199:44303
I20240502 01:59:17.242367 29369 tablet_copy_service.cc:140] P 468dd27456c64d67b22e8a58af93d93d: Received BeginTabletCopySession request for tablet 12420fbcddc5446b857d2e6a01f40aaa from peer 68e5b1eeb04448fea818a14991eab87c ({username='slave'} at 127.19.43.196:55625)
I20240502 01:59:17.242463 29369 tablet_copy_service.cc:161] P 468dd27456c64d67b22e8a58af93d93d: Beginning new tablet copy session on tablet 12420fbcddc5446b857d2e6a01f40aaa from peer 68e5b1eeb04448fea818a14991eab87c at {username='slave'} at 127.19.43.196:55625: session id = 68e5b1eeb04448fea818a14991eab87c-12420fbcddc5446b857d2e6a01f40aaa
I20240502 01:59:17.243432 29369 tablet_copy_source_session.cc:205] T 12420fbcddc5446b857d2e6a01f40aaa P 468dd27456c64d67b22e8a58af93d93d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:17.244335 8069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12420fbcddc5446b857d2e6a01f40aaa. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 01:59:17.246253 26595 consensus_peers.cc:485] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:17.250730 8069 tablet_copy_client.cc:783] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:17.250977 8069 tablet_copy_client.cc:647] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:17.252032 8069 tablet_copy_client.cc:523] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:17.253760 8069 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
W20240502 01:59:17.258190 29016 consensus_peers.cc:485] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:17.264729 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:17.267769 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:17.269877 29350 raft_consensus.cc:1203] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
W20240502 01:59:17.273998 29286 consensus_peers.cc:485] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.274885 26595 consensus_peers.cc:485] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.275027 29150 consensus_peers.cc:485] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:17.280297 8069 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:17.280498 8069 tablet_bootstrap.cc:492] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: Bootstrap complete.
I20240502 01:59:17.280642 8069 ts_tablet_manager.cc:1367] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20240502 01:59:17.281013 8069 raft_consensus.cc:348] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:17.281111 8069 raft_consensus.cc:729] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:17.281241 8069 consensus_queue.cc:260] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:17.283090 8069 ts_tablet_manager.cc:1398] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:17.283948 29196 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:17.284109 29369 tablet_copy_service.cc:342] P 468dd27456c64d67b22e8a58af93d93d: Request end of tablet copy session 68e5b1eeb04448fea818a14991eab87c-12420fbcddc5446b857d2e6a01f40aaa received from {username='slave'} at 127.19.43.196:55625
I20240502 01:59:17.284183 29369 tablet_copy_service.cc:433] P 468dd27456c64d67b22e8a58af93d93d: ending tablet copy session 68e5b1eeb04448fea818a14991eab87c-12420fbcddc5446b857d2e6a01f40aaa on tablet 12420fbcddc5446b857d2e6a01f40aaa with peer 68e5b1eeb04448fea818a14991eab87c
I20240502 01:59:17.284606 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:17.288376 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:17.288601 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:17.312189 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
W20240502 01:59:17.316414 29016 consensus_peers.cc:485] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:17.327018 26794 raft_consensus.cc:1203] T 398742b2cb3e41f1bc16fa37a5168809 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 01:59:17.357578 26862 consensus_peers.cc:485] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:17.387411 26595 consensus_peers.cc:485] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:17.459089 26661 raft_consensus.cc:1203] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:17.472724 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:17.476514 29196 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:17.477478 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:17.481477 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:17.485575 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:17.494323 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:17.506664 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:17.508134 26794 raft_consensus.cc:1203] T 4caef235274947e48765e1c11669112a P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 01:59:17.519826 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.537678 26661 raft_consensus.cc:1203] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 01:59:17.555344 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.555478 26927 raft_consensus.cc:1203] T 11a922c5a8994e92a4112d89ce8a9f3a P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:17.678755 26927 raft_consensus.cc:1203] T 68f002fcea6f4378954808dbfb5f10e6 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:17.681423 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:17.682075 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:17.682121 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:17.687641 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:17.687736 29196 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:17.695144 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:17.720319 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:17.823096 27064 raft_consensus.cc:1203] T 12420fbcddc5446b857d2e6a01f40aaa P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 01:59:17.824007 7852 raft_consensus.cc:482] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.824115 7852 raft_consensus.cc:504] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:17.824558 7852 leader_election.cc:288] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:17.825220 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:17.825737 7221 raft_consensus.cc:482] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
W20240502 01:59:17.825752 29016 leader_election.cc:334] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.825846 29015 leader_election.cc:302] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d18a019013487ab801599cfb80e4b4; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:17.825829 7221 raft_consensus.cc:504] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:17.825976 7852 raft_consensus.cc:2726] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.826187 7221 leader_election.cc:288] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:17.826535 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
W20240502 01:59:17.827997 29150 leader_election.cc:334] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.828079 29150 leader_election.cc:302] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df; no voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:17.828215 7221 raft_consensus.cc:2726] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.850747 7221 raft_consensus.cc:482] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.850849 7221 raft_consensus.cc:504] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:17.851212 7221 leader_election.cc:288] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:17.852093 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
W20240502 01:59:17.853876 29150 leader_election.cc:334] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.853951 29150 leader_election.cc:302] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df; no voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:17.854204 7221 raft_consensus.cc:2726] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.864002 7852 raft_consensus.cc:482] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.864089 7852 raft_consensus.cc:504] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:17.864416 7852 leader_election.cc:288] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:17.864856 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
W20240502 01:59:17.866458 29016 leader_election.cc:334] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.866533 29016 leader_election.cc:302] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124
I20240502 01:59:17.866681 7852 raft_consensus.cc:2726] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.870935 8086 raft_consensus.cc:482] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.871057 8086 raft_consensus.cc:504] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:17.871404 8086 leader_election.cc:288] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:17.872022 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:17.872855 7221 raft_consensus.cc:482] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.872934 7221 raft_consensus.cc:504] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:17.873258 7221 leader_election.cc:288] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:17.874316 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
W20240502 01:59:17.874781 29150 leader_election.cc:334] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.874944 29150 leader_election.cc:302] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df; no voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:17.875139 7221 raft_consensus.cc:2726] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 01:59:17.875798 29286 leader_election.cc:334] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.875869 29286 leader_election.cc:302] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d; no voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:17.876397 8086 raft_consensus.cc:2726] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.883862 29196 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 01:59:17.885118 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:17.886518 7755 raft_consensus.cc:482] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.886613 7755 raft_consensus.cc:504] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:17.886972 7755 leader_election.cc:288] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303)
I20240502 01:59:17.888036 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
W20240502 01:59:17.889926 26862 leader_election.cc:334] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.890002 26862 leader_election.cc:302] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e; no voters: 468dd27456c64d67b22e8a58af93d93d, 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:17.890138 7755 raft_consensus.cc:2726] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.893781 29328 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:17.895372 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:17.896798 26909 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:17.901124 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:17.910084 8196 raft_consensus.cc:482] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.910182 8196 raft_consensus.cc:504] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:17.910501 8196 leader_election.cc:288] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:17.911476 26794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "998b57e2badc4d779c586c6579149114" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
W20240502 01:59:17.912807 26595 leader_election.cc:334] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.912880 26595 leader_election.cc:302] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 998b57e2badc4d779c586c6579149114; no voters: 7c693d0933c64d9babc07f404df45cd1, b290c1bf893444078ed721f25995d124
I20240502 01:59:17.918104 7755 raft_consensus.cc:482] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.918183 7755 raft_consensus.cc:504] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:17.918479 7755 leader_election.cc:288] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:17.918540 8196 raft_consensus.cc:2726] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240502 01:59:17.919382 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
W20240502 01:59:17.919682 26862 leader_election.cc:334] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.919929 26859 leader_election.cc:302] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e; no voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:17.920148 7755 raft_consensus.cc:2726] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.921470 26773 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:17.935680 4345 raft_consensus.cc:482] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.935772 4345 raft_consensus.cc:504] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:17.936071 4345 leader_election.cc:288] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
W20240502 01:59:17.939015 26998 leader_election.cc:334] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.939206 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:17.939594 26997 leader_election.cc:302] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:17.939852 8195 raft_consensus.cc:482] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.939929 8195 raft_consensus.cc:504] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.940282 8195 leader_election.cc:288] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:17.940676 4345 raft_consensus.cc:2726] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.940963 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc69c007af34db18b67f7d6c4f20d80" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:17.941083 26658 raft_consensus.cc:2454] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:17.941402 26729 leader_election.cc:302] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:17.941587 8195 raft_consensus.cc:2781] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:17.941648 8195 raft_consensus.cc:482] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.941682 8195 raft_consensus.cc:3037] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:17.942984 8195 raft_consensus.cc:504] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.943300 8195 leader_election.cc:288] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
W20240502 01:59:17.943712 26730 leader_election.cc:334] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.943769 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc69c007af34db18b67f7d6c4f20d80" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:17.943864 26658 raft_consensus.cc:3037] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:17.944754 26658 raft_consensus.cc:2454] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 2.
I20240502 01:59:17.945171 26729 leader_election.cc:302] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
W20240502 01:59:17.945554 26730 leader_election.cc:334] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.946056 8195 raft_consensus.cc:2781] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:17.946136 8195 raft_consensus.cc:686] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [term 2 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:17.946257 8195 consensus_queue.cc:237] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:17.947005 7855 raft_consensus.cc:482] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.947090 7855 raft_consensus.cc:504] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:17.947475 7855 leader_election.cc:288] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:17.948065 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
W20240502 01:59:17.948351 29016 leader_election.cc:334] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.948482 29015 leader_election.cc:302] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d18a019013487ab801599cfb80e4b4; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:17.948616 7855 raft_consensus.cc:2726] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:17.948818 26510 catalog_manager.cc:5484] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:17.975939 8086 raft_consensus.cc:482] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.976037 8086 raft_consensus.cc:504] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:17.976342 8086 leader_election.cc:288] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:17.977063 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:17.977183 26927 raft_consensus.cc:2454] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
I20240502 01:59:17.977459 29286 leader_election.cc:302] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:17.977747 8086 raft_consensus.cc:2781] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:17.977795 8086 raft_consensus.cc:482] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:17.977828 8086 raft_consensus.cc:3037] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:17.978111 29286 leader_election.cc:334] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.979068 8086 raft_consensus.cc:504] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:17.979385 8086 leader_election.cc:288] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 election: Requested vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:17.979629 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5f8bfeddd1443da789f67949830f99" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:17.979723 26927 raft_consensus.cc:3037] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:17.980571 26927 raft_consensus.cc:2454] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 2.
I20240502 01:59:17.980815 29286 leader_election.cc:302] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:17.981068 8086 raft_consensus.cc:2781] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:17.981137 8086 raft_consensus.cc:686] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [term 2 LEADER]: Becoming Leader. State: Replica: 468dd27456c64d67b22e8a58af93d93d, State: Running, Role: LEADER
I20240502 01:59:17.981252 8086 consensus_queue.cc:237] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
W20240502 01:59:17.982110 29286 leader_election.cc:334] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:17.995157 26510 catalog_manager.cc:5484] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 468dd27456c64d67b22e8a58af93d93d (127.19.43.199). New cstate: current_term: 2 leader_uuid: "468dd27456c64d67b22e8a58af93d93d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:18.020465 4345 raft_consensus.cc:482] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.020565 4345 raft_consensus.cc:504] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:18.020982 4345 leader_election.cc:288] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.021744 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
W20240502 01:59:18.022487 26995 leader_election.cc:341] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Not found: Tablet not found: e66b19b5551b4011bf1bbb6234a2aba3
I20240502 01:59:18.022555 26995 leader_election.cc:302] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 468dd27456c64d67b22e8a58af93d93d, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:18.022701 4345 raft_consensus.cc:2726] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 01:59:18.023202 26998 leader_election.cc:334] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.023461 7447 raft_consensus.cc:482] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.023550 7447 raft_consensus.cc:504] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:18.023960 7447 leader_election.cc:288] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.024534 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:18.024649 27062 raft_consensus.cc:2454] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
W20240502 01:59:18.025009 29283 leader_election.cc:341] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Not found: Tablet not found: e66b19b5551b4011bf1bbb6234a2aba3
W20240502 01:59:18.025626 29286 leader_election.cc:334] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.025682 29286 leader_election.cc:302] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:18.025822 7447 raft_consensus.cc:2726] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:18.033840 8272 raft_consensus.cc:482] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.033977 8272 raft_consensus.cc:504] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.034389 8272 leader_election.cc:288] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:18.034837 26927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
W20240502 01:59:18.035239 29150 leader_election.cc:334] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.035306 29150 leader_election.cc:302] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df; no voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:18.035812 8086 raft_consensus.cc:482] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.035885 8272 raft_consensus.cc:2726] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:18.035902 8086 raft_consensus.cc:504] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.036237 8086 leader_election.cc:288] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.036628 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
W20240502 01:59:18.038774 29286 leader_election.cc:334] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.038841 29286 leader_election.cc:302] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.039034 8086 raft_consensus.cc:2726] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:18.043516 8195 raft_consensus.cc:482] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.043612 8195 raft_consensus.cc:504] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.043958 8195 leader_election.cc:288] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
W20240502 01:59:18.045413 26730 leader_election.cc:334] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.045574 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:18.046123 26730 leader_election.cc:302] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b290c1bf893444078ed721f25995d124; no voters: 7c693d0933c64d9babc07f404df45cd1, 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:18.046370 6828 raft_consensus.cc:2726] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:18.068416 8272 raft_consensus.cc:482] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.068502 8272 raft_consensus.cc:504] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.068836 8272 leader_election.cc:288] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.069166 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:18.069295 29080 raft_consensus.cc:2454] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:18.069805 29147 leader_election.cc:302] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:18.070581 8272 raft_consensus.cc:2781] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.070649 8272 raft_consensus.cc:482] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.070696 8272 raft_consensus.cc:3037] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.070796 29150 leader_election.cc:334] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.071794 8272 raft_consensus.cc:504] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.072090 8272 leader_election.cc:288] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.072844 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d6c35112da74b34b7c71318c61fda5e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:18.072952 29080 raft_consensus.cc:3037] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.073060 29150 leader_election.cc:334] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.073797 29080 raft_consensus.cc:2454] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:18.074174 29147 leader_election.cc:302] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.074314 8272 raft_consensus.cc:2781] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.074388 8272 raft_consensus.cc:686] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:18.074517 8272 consensus_queue.cc:237] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.076368 7755 raft_consensus.cc:482] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.076455 7755 raft_consensus.cc:504] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.076793 7755 leader_election.cc:288] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.077162 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:18.077270 29215 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:18.077579 26861 leader_election.cc:302] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:18.077785 7755 raft_consensus.cc:2781] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.077836 7755 raft_consensus.cc:482] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.077872 7755 raft_consensus.cc:3037] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.078963 7755 raft_consensus.cc:504] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.079005 8272 raft_consensus.cc:482] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.079083 8272 raft_consensus.cc:504] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.079283 7755 leader_election.cc:288] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.079409 8272 leader_election.cc:288] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
W20240502 01:59:18.080376 29150 leader_election.cc:334] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.081643 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.081665 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:18.081777 29080 raft_consensus.cc:2454] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:18.082060 29147 leader_election.cc:302] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.082388 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee632f93fe234584814e6aa7b6a63bc0" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:18.082484 29215 raft_consensus.cc:3037] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.083431 26862 leader_election.cc:334] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.083499 29215 raft_consensus.cc:2454] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 2.
I20240502 01:59:18.083828 8272 raft_consensus.cc:2781] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.083839 26861 leader_election.cc:302] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:18.083891 8272 raft_consensus.cc:482] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.083933 8272 raft_consensus.cc:3037] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.084754 8272 raft_consensus.cc:504] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.085078 8272 leader_election.cc:288] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Requested vote from peers f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.085384 8151 raft_consensus.cc:2781] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.085464 8151 raft_consensus.cc:686] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [term 2 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:18.085593 8151 consensus_queue.cc:237] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
W20240502 01:59:18.086051 29150 leader_election.cc:334] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.086372 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463056ff0e244d2fa7dd6c4c70f3eded" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:18.086478 29080 raft_consensus.cc:3037] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.087421 29080 raft_consensus.cc:2454] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:18.087709 29147 leader_election.cc:302] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1
W20240502 01:59:18.087786 26862 leader_election.cc:334] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.088068 8272 raft_consensus.cc:2781] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.088165 8272 raft_consensus.cc:686] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:18.088282 8272 consensus_queue.cc:237] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.088902 26512 catalog_manager.cc:5484] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 2 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:18.093711 26510 catalog_manager.cc:5484] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:18.098359 26510 catalog_manager.cc:5484] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
W20240502 01:59:18.106307 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.113806 26909 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 01:59:18.115878 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:18.119393 29196 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:18.119624 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:18.125305 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:18.139432 7755 raft_consensus.cc:482] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.139520 7755 raft_consensus.cc:504] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.139871 7755 leader_election.cc:288] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.140563 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:18.140678 29350 raft_consensus.cc:2454] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:18.140987 26859 leader_election.cc:302] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:18.141362 7755 raft_consensus.cc:2781] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.141413 7755 raft_consensus.cc:482] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.141446 7755 raft_consensus.cc:3037] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.142508 26862 leader_election.cc:334] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.142495 7755 raft_consensus.cc:504] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.143079 7755 leader_election.cc:288] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.146747 26773 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 01:59:18.148331 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab0d9a2f05547e788258e5a486a7dd2" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:18.148437 29350 raft_consensus.cc:3037] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.148836 8195 raft_consensus.cc:482] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.148906 8195 raft_consensus.cc:504] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.149291 8195 leader_election.cc:288] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.149421 29350 raft_consensus.cc:2454] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 2.
I20240502 01:59:18.150074 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:18.150214 29080 raft_consensus.cc:2454] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 1.
I20240502 01:59:18.150547 26727 leader_election.cc:302] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:18.150738 8195 raft_consensus.cc:2781] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.150785 8195 raft_consensus.cc:482] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.150820 8195 raft_consensus.cc:3037] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.151816 8195 raft_consensus.cc:504] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.152088 8195 leader_election.cc:288] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.153203 26859 leader_election.cc:302] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:18.154384 29058 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 01:59:18.158283 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c5307ea58d40dc80079b45f17db1d2" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4"
I20240502 01:59:18.158386 29080 raft_consensus.cc:3037] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.159348 29080 raft_consensus.cc:2454] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 2.
I20240502 01:59:18.159509 8151 raft_consensus.cc:2781] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.159644 8151 raft_consensus.cc:686] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [term 2 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:18.159762 8151 consensus_queue.cc:237] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.162001 26510 catalog_manager.cc:5484] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 2 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
W20240502 01:59:18.162015 26730 leader_election.cc:334] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.162521 26727 leader_election.cc:302] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b290c1bf893444078ed721f25995d124, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:18.164641 6828 raft_consensus.cc:2781] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.164800 6828 raft_consensus.cc:686] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [term 2 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:18.164923 6828 consensus_queue.cc:237] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.167488 26510 catalog_manager.cc:5484] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:59:18.167670 26862 leader_election.cc:334] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.169085 26730 leader_election.cc:334] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.178381 6980 raft_consensus.cc:482] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.178468 6980 raft_consensus.cc:504] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.178812 6980 leader_election.cc:288] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.179039 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
W20240502 01:59:18.179860 26998 leader_election.cc:334] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.179925 26998 leader_election.cc:302] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.180107 6980 raft_consensus.cc:2726] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:18.194013 8194 raft_consensus.cc:482] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.194149 8194 raft_consensus.cc:504] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.194454 8194 leader_election.cc:288] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.194730 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:18.194850 27062 raft_consensus.cc:2454] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:18.195240 29150 leader_election.cc:302] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:18.195475 8272 raft_consensus.cc:2781] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.195535 8272 raft_consensus.cc:482] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.195576 8272 raft_consensus.cc:3037] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.196311 29150 leader_election.cc:334] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.196605 8272 raft_consensus.cc:504] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.196918 8272 leader_election.cc:288] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.197221 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3d890af39194df197564cb91e80297e" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:18.197332 27062 raft_consensus.cc:3037] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.197856 29150 leader_election.cc:334] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.198390 27062 raft_consensus.cc:2454] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:18.198686 29150 leader_election.cc:302] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.198968 8272 raft_consensus.cc:2781] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.199047 8272 raft_consensus.cc:686] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:18.199167 8272 consensus_queue.cc:237] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.201958 26510 catalog_manager.cc:5484] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:18.245391 8151 raft_consensus.cc:482] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.245504 8151 raft_consensus.cc:504] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.245859 8151 leader_election.cc:288] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.246196 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:18.246331 29215 raft_consensus.cc:2454] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:18.246579 26861 leader_election.cc:302] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:18.246748 8151 raft_consensus.cc:2781] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.246822 8151 raft_consensus.cc:482] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.246858 8151 raft_consensus.cc:3037] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.247081 26862 leader_election.cc:334] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.247860 8151 raft_consensus.cc:504] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.248189 8151 leader_election.cc:288] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.248858 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86bd546634a948069a9f97673dfc188c" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:18.248965 29215 raft_consensus.cc:3037] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.250052 29215 raft_consensus.cc:2454] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 2.
I20240502 01:59:18.250479 26861 leader_election.cc:302] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 842c1b011b5d4c77bfd9b9591111502e; no voters:
I20240502 01:59:18.250707 8151 raft_consensus.cc:2781] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.250787 8151 raft_consensus.cc:686] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [term 2 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:18.250907 8151 consensus_queue.cc:237] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
W20240502 01:59:18.251013 26862 leader_election.cc:334] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.253121 26510 catalog_manager.cc:5484] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 2 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:18.276408 8272 raft_consensus.cc:482] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.276507 8272 raft_consensus.cc:504] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.276881 8272 leader_election.cc:288] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.277719 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:18.277856 27062 raft_consensus.cc:2454] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:18.278537 29150 leader_election.cc:302] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:18.279042 8272 raft_consensus.cc:2781] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.279103 8272 raft_consensus.cc:482] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.279150 8272 raft_consensus.cc:3037] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.280138 8272 raft_consensus.cc:504] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.280463 8272 leader_election.cc:288] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Requested vote from peers 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.280665 8272 raft_consensus.cc:482] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
W20240502 01:59:18.280718 29150 leader_election.cc:334] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.280748 8272 raft_consensus.cc:504] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.281100 8272 leader_election.cc:288] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.281744 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb13643f981847c4bb480b74a4764c4c" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:18.281867 27062 raft_consensus.cc:3037] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.281909 29350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d" is_pre_election: true
I20240502 01:59:18.282052 7755 raft_consensus.cc:482] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.282063 29350 raft_consensus.cc:2454] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 1.
I20240502 01:59:18.282128 7755 raft_consensus.cc:504] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.282341 29147 leader_election.cc:302] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:59:18.282475 7755 leader_election.cc:288] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:18.282526 8272 raft_consensus.cc:2781] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.282567 8272 raft_consensus.cc:482] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.282603 8272 raft_consensus.cc:3037] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.282900 27062 raft_consensus.cc:2454] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:18.283226 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124" is_pre_election: true
I20240502 01:59:18.283340 26795 raft_consensus.cc:2454] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 1.
I20240502 01:59:18.283922 26861 leader_election.cc:302] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:18.283963 8272 raft_consensus.cc:504] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.284446 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98e2f3fb2ea4241a16db8b4bb60b4b8" candidate_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "468dd27456c64d67b22e8a58af93d93d"
I20240502 01:59:18.284546 29348 raft_consensus.cc:3037] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.284807 26862 leader_election.cc:334] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.284978 7755 raft_consensus.cc:2781] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.285017 7755 raft_consensus.cc:482] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.285048 7755 raft_consensus.cc:3037] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.285503 29348 raft_consensus.cc:2454] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d83fdd1e6e04cf5a5602994d1dbc0df in term 2.
I20240502 01:59:18.285863 29147 leader_election.cc:302] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 468dd27456c64d67b22e8a58af93d93d; no voters:
I20240502 01:59:18.286053 8194 raft_consensus.cc:2781] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.286082 7755 raft_consensus.cc:504] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.286139 8194 raft_consensus.cc:686] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:18.286181 29150 leader_election.cc:302] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, 68e5b1eeb04448fea818a14991eab87c; no voters:
I20240502 01:59:18.286268 8194 consensus_queue.cc:237] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } }
I20240502 01:59:18.286383 7755 leader_election.cc:288] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), b290c1bf893444078ed721f25995d124 (127.19.43.194:38007)
I20240502 01:59:18.286391 7221 raft_consensus.cc:2781] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.286564 7221 raft_consensus.cc:686] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Becoming Leader. State: Replica: 0d83fdd1e6e04cf5a5602994d1dbc0df, State: Running, Role: LEADER
I20240502 01:59:18.286689 7221 consensus_queue.cc:237] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
W20240502 01:59:18.287633 29150 leader_election.cc:334] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.287748 29150 leader_election.cc:334] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.287997 26862 leader_election.cc:334] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.287997 26795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9713e696e4aabb13f7009bf9ca490" candidate_uuid: "842c1b011b5d4c77bfd9b9591111502e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b290c1bf893444078ed721f25995d124"
I20240502 01:59:18.288097 26795 raft_consensus.cc:3037] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.289078 26795 raft_consensus.cc:2454] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842c1b011b5d4c77bfd9b9591111502e in term 2.
I20240502 01:59:18.289356 26861 leader_election.cc:302] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, b290c1bf893444078ed721f25995d124; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.289732 7755 raft_consensus.cc:2781] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.289815 7755 raft_consensus.cc:686] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [term 2 LEADER]: Becoming Leader. State: Replica: 842c1b011b5d4c77bfd9b9591111502e, State: Running, Role: LEADER
I20240502 01:59:18.289928 7755 consensus_queue.cc:237] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.290508 26515 catalog_manager.cc:5484] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } }
I20240502 01:59:18.290614 8272 leader_election.cc:288] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: Requested vote from peers 468dd27456c64d67b22e8a58af93d93d (127.19.43.199:44303), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.292387 26510 catalog_manager.cc:5484] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195). New cstate: current_term: 2 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
W20240502 01:59:18.299439 29150 leader_election.cc:334] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.302495 26515 catalog_manager.cc:5484] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198). New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:18.305713 7852 raft_consensus.cc:482] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.305812 7852 raft_consensus.cc:504] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.306249 7852 leader_election.cc:288] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:18.308343 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e" is_pre_election: true
I20240502 01:59:18.308456 26930 raft_consensus.cc:2454] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:18.308903 29016 leader_election.cc:302] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:18.309211 7855 raft_consensus.cc:2781] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.309264 7855 raft_consensus.cc:482] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.309298 7855 raft_consensus.cc:3037] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.309531 29016 leader_election.cc:334] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.310454 7855 raft_consensus.cc:504] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.310809 7855 leader_election.cc:288] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379)
I20240502 01:59:18.311908 26930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f518620fa0894706b44e22c7f3c51774" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "842c1b011b5d4c77bfd9b9591111502e"
I20240502 01:59:18.312012 26930 raft_consensus.cc:3037] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:18.312026 29016 leader_election.cc:334] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.312891 26930 raft_consensus.cc:2454] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 2.
I20240502 01:59:18.313187 7855 raft_consensus.cc:482] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.313226 29016 leader_election.cc:302] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 842c1b011b5d4c77bfd9b9591111502e, f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.313270 7855 raft_consensus.cc:504] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.313421 7852 raft_consensus.cc:2781] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.313567 7855 leader_election.cc:288] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.313583 7852 raft_consensus.cc:686] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [term 2 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:18.313737 7852 consensus_queue.cc:237] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } }
I20240502 01:59:18.313880 6980 raft_consensus.cc:482] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.313959 6980 raft_consensus.cc:504] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:18.314319 6980 leader_election.cc:288] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:18.314915 29080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "68e5b1eeb04448fea818a14991eab87c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3d18a019013487ab801599cfb80e4b4" is_pre_election: true
I20240502 01:59:18.315174 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:18.315286 29215 raft_consensus.cc:2454] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:18.315541 29015 leader_election.cc:302] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:18.315670 7855 raft_consensus.cc:2781] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.315713 7855 raft_consensus.cc:482] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.315747 7855 raft_consensus.cc:3037] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.316200 7852 raft_consensus.cc:482] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.316269 7852 raft_consensus.cc:504] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:18.316553 7852 leader_election.cc:288] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.316627 7855 raft_consensus.cc:504] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.316916 7855 leader_election.cc:288] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Requested vote from peers 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103), 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259)
I20240502 01:59:18.316910 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" is_pre_election: true
I20240502 01:59:18.317019 29215 raft_consensus.cc:2454] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
W20240502 01:59:18.317054 26998 leader_election.cc:334] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.317114 26998 leader_election.cc:302] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c; no voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:18.317302 6980 raft_consensus.cc:2726] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 01:59:18.317623 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab01aa0450bb4ef388b49e38369ab20b" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:18.317658 29015 leader_election.cc:302] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
I20240502 01:59:18.317720 29215 raft_consensus.cc:3037] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.317867 7855 raft_consensus.cc:2781] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.317921 7855 raft_consensus.cc:482] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.317956 7855 raft_consensus.cc:3037] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.318710 29215 raft_consensus.cc:2454] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 2.
I20240502 01:59:18.318712 7855 raft_consensus.cc:504] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:18.319133 29015 leader_election.cc:302] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
W20240502 01:59:18.319200 29016 leader_election.cc:334] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.319310 29016 leader_election.cc:334] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.319394 29016 leader_election.cc:334] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.319573 7852 raft_consensus.cc:2781] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.319568 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713002b695184076816d7748c052ce59" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df"
I20240502 01:59:18.319653 29215 raft_consensus.cc:3037] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.319656 7852 raft_consensus.cc:686] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [term 2 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:18.319761 7852 consensus_queue.cc:237] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:18.320503 29215 raft_consensus.cc:2454] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 2.
I20240502 01:59:18.321702 7855 leader_election.cc:288] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:18.323415 29015 leader_election.cc:302] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d83fdd1e6e04cf5a5602994d1dbc0df, f3d18a019013487ab801599cfb80e4b4; no voters:
W20240502 01:59:18.325227 29016 leader_election.cc:334] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:18.325536 26730 consensus_peers.cc:485] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.325850 7855 raft_consensus.cc:2781] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.325984 7855 raft_consensus.cc:686] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [term 2 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:18.326160 7855 consensus_queue.cc:237] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:18.327529 26510 catalog_manager.cc:5484] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 2 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } }
I20240502 01:59:18.332012 26510 catalog_manager.cc:5484] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 2 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } }
I20240502 01:59:18.332223 26510 catalog_manager.cc:5484] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 2 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } }
I20240502 01:59:18.335100 8370 raft_consensus.cc:482] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.335186 8370 raft_consensus.cc:504] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
W20240502 01:59:18.336918 29016 leader_election.cc:334] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.341539 8370 leader_election.cc:288] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:18.341934 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
I20240502 01:59:18.342075 27062 raft_consensus.cc:2454] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 1.
I20240502 01:59:18.342409 29016 leader_election.cc:302] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.342692 8370 raft_consensus.cc:2781] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 01:59:18.342785 8370 raft_consensus.cc:482] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.342870 8370 raft_consensus.cc:3037] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.343896 8370 raft_consensus.cc:504] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:18.345741 8195 raft_consensus.cc:482] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.345837 8195 raft_consensus.cc:504] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
I20240502 01:59:18.346223 8195 leader_election.cc:288] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:18.346239 8370 leader_election.cc:288] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635)
I20240502 01:59:18.346881 27062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca8563ff68c4fa590c17ef58e67afa0" candidate_uuid: "f3d18a019013487ab801599cfb80e4b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c"
I20240502 01:59:18.346873 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114" is_pre_election: true
I20240502 01:59:18.346987 27062 raft_consensus.cc:3037] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:18.347000 26658 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 2.
I20240502 01:59:18.347245 26729 leader_election.cc:302] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:18.347388 8195 raft_consensus.cc:2781] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240502 01:59:18.347451 8195 raft_consensus.cc:482] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:18.347491 8195 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:18.347607 26730 leader_election.cc:334] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.347981 27062 raft_consensus.cc:2454] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d18a019013487ab801599cfb80e4b4 in term 2.
I20240502 01:59:18.348210 8195 raft_consensus.cc:504] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
W20240502 01:59:18.348266 29016 leader_election.cc:334] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.348405 29016 leader_election.cc:302] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68e5b1eeb04448fea818a14991eab87c, f3d18a019013487ab801599cfb80e4b4; no voters: 7c693d0933c64d9babc07f404df45cd1
I20240502 01:59:18.348577 8370 raft_consensus.cc:2781] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:18.348584 8195 leader_election.cc:288] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 3 election: Requested vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929)
I20240502 01:59:18.348697 8370 raft_consensus.cc:686] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [term 2 LEADER]: Becoming Leader. State: Replica: f3d18a019013487ab801599cfb80e4b4, State: Running, Role: LEADER
I20240502 01:59:18.348822 8370 consensus_queue.cc:237] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:18.348906 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a326a7899c064c95b426aae61d65208f" candidate_uuid: "b290c1bf893444078ed721f25995d124" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "998b57e2badc4d779c586c6579149114"
I20240502 01:59:18.348999 26658 raft_consensus.cc:3037] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:18.349853 26658 raft_consensus.cc:2454] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b290c1bf893444078ed721f25995d124 in term 3.
I20240502 01:59:18.350154 26729 leader_election.cc:302] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998b57e2badc4d779c586c6579149114, b290c1bf893444078ed721f25995d124; no voters:
I20240502 01:59:18.350497 6828 raft_consensus.cc:2781] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:18.350577 6828 raft_consensus.cc:686] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [term 3 LEADER]: Becoming Leader. State: Replica: b290c1bf893444078ed721f25995d124, State: Running, Role: LEADER
I20240502 01:59:18.350697 6828 consensus_queue.cc:237] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } }
W20240502 01:59:18.350726 26730 leader_election.cc:334] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.351132 26510 catalog_manager.cc:5484] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: term changed from 1 to 2, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to f3d18a019013487ab801599cfb80e4b4 (127.19.43.197). New cstate: current_term: 2 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:18.353111 26508 catalog_manager.cc:5484] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 reported cstate change: term changed from 2 to 3, leader changed from 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200) to b290c1bf893444078ed721f25995d124 (127.19.43.194). New cstate: current_term: 3 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } }
I20240502 01:59:18.365731 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:18.368644 29196 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:18.369693 26930 raft_consensus.cc:1261] T 6a5f8bfeddd1443da789f67949830f99 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Refusing update from remote peer 468dd27456c64d67b22e8a58af93d93d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:18.370195 8315 consensus_queue.cc:1035] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:18.381803 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:18.382673 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:18.388357 26909 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:18.390450 26773 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
W20240502 01:59:18.396590 29286 consensus_peers.cc:485] T 6a5f8bfeddd1443da789f67949830f99 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.402472 29058 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
W20240502 01:59:18.457036 29150 consensus_peers.cc:485] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.462381 26795 consensus_queue.cc:237] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } }
I20240502 01:59:18.463585 29348 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:18.463629 29215 raft_consensus.cc:1261] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 01:59:18.463933 6828 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:18.464107 8195 consensus_queue.cc:1035] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:18.465797 8318 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:18.466217 29348 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:18.466204 29215 raft_consensus.cc:2932] T b265eec55ee1406b9ff99c159304fa65 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:18.469318 26509 catalog_manager.cc:5484] T b265eec55ee1406b9ff99c159304fa65 P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 6 to 7, VOTER 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } }
I20240502 01:59:18.472281 26909 tablet_service.cc:1514] Processing DeleteTablet for tablet b265eec55ee1406b9ff99c159304fa65 with delete_type TABLET_DATA_TOMBSTONED (TS 842c1b011b5d4c77bfd9b9591111502e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:59716
I20240502 01:59:18.472985 8413 tablet_replica.cc:330] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: stopping tablet replica
I20240502 01:59:18.473207 8413 raft_consensus.cc:2229] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:18.473745 8413 raft_consensus.cc:2258] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:18.474401 8413 ts_tablet_manager.cc:1875] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:18.475361 29080 raft_consensus.cc:1261] T 7d6c35112da74b34b7c71318c61fda5e P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.475790 8272 consensus_queue.cc:1035] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.477391 8413 ts_tablet_manager.cc:1888] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 01:59:18.477499 8413 log.cc:1196] T b265eec55ee1406b9ff99c159304fa65 P 842c1b011b5d4c77bfd9b9591111502e: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/wals/b265eec55ee1406b9ff99c159304fa65
I20240502 01:59:18.478319 26502 catalog_manager.cc:4830] TS 842c1b011b5d4c77bfd9b9591111502e (127.19.43.195:40379): tablet b265eec55ee1406b9ff99c159304fa65 (table rebalance_test_table_17 [id=8417deae11f14e4e8e5d8d5fddd562ed]) successfully deleted
W20240502 01:59:18.505798 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.506343 29079 raft_consensus.cc:1261] T 463056ff0e244d2fa7dd6c4c70f3eded P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.506660 8272 consensus_queue.cc:1035] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:18.509495 29150 consensus_peers.cc:485] T 463056ff0e244d2fa7dd6c4c70f3eded P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.549909 26658 raft_consensus.cc:1261] T 8fc69c007af34db18b67f7d6c4f20d80 P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.550590 6828 consensus_queue.cc:1035] T 8fc69c007af34db18b67f7d6c4f20d80 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:18.570677 26862 consensus_peers.cc:485] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:18.579349 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.604539 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:18.609568 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:18.611697 29196 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:18.614670 26909 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:18.627552 29058 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 01:59:18.628934 29080 raft_consensus.cc:1261] T 59c5307ea58d40dc80079b45f17db1d2 P f3d18a019013487ab801599cfb80e4b4 [term 2 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.630184 8318 consensus_queue.cc:1035] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.634863 26773 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:18.636049 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
W20240502 01:59:18.649600 29150 consensus_peers.cc:485] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:18.671530 26730 consensus_peers.cc:485] T 59c5307ea58d40dc80079b45f17db1d2 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:18.683354 26862 consensus_peers.cc:485] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.697290 27062 raft_consensus.cc:1261] T c3d890af39194df197564cb91e80297e P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.697660 7221 consensus_queue.cc:1035] T c3d890af39194df197564cb91e80297e P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.705873 29215 raft_consensus.cc:1261] T ee632f93fe234584814e6aa7b6a63bc0 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:18.706291 8151 consensus_queue.cc:1035] T ee632f93fe234584814e6aa7b6a63bc0 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:18.720073 29348 raft_consensus.cc:1261] T fab0d9a2f05547e788258e5a486a7dd2 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.720485 7755 consensus_queue.cc:1035] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:18.723899 26862 consensus_peers.cc:485] T fab0d9a2f05547e788258e5a486a7dd2 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.731709 29215 raft_consensus.cc:1261] T 713002b695184076816d7748c052ce59 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.732093 7855 consensus_queue.cc:1035] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:18.735677 29016 consensus_peers.cc:485] T 713002b695184076816d7748c052ce59 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:18.736514 29016 consensus_peers.cc:485] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:18.765151 29150 consensus_peers.cc:485] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:18.808542 29016 consensus_peers.cc:485] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.812280 26795 raft_consensus.cc:1261] T f5d9713e696e4aabb13f7009bf9ca490 P b290c1bf893444078ed721f25995d124 [term 2 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.813089 8151 consensus_queue.cc:1035] T f5d9713e696e4aabb13f7009bf9ca490 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:18.813733 29150 consensus_peers.cc:485] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.822532 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
W20240502 01:59:18.828559 26862 consensus_peers.cc:485] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.829016 29196 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
W20240502 01:59:18.830897 26730 consensus_peers.cc:485] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.837002 29058 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 01:59:18.841827 26909 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:18.845837 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:18.849479 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:18.853283 26658 raft_consensus.cc:1261] T a326a7899c064c95b426aae61d65208f P 998b57e2badc4d779c586c6579149114 [term 3 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:18.853644 6828 consensus_queue.cc:1035] T a326a7899c064c95b426aae61d65208f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:18.854684 29215 raft_consensus.cc:1261] T ab01aa0450bb4ef388b49e38369ab20b P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.855036 7855 consensus_queue.cc:1035] T ab01aa0450bb4ef388b49e38369ab20b P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.859460 26930 raft_consensus.cc:1261] T f518620fa0894706b44e22c7f3c51774 P 842c1b011b5d4c77bfd9b9591111502e [term 2 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.860056 7855 consensus_queue.cc:1035] T f518620fa0894706b44e22c7f3c51774 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.864205 29215 raft_consensus.cc:1261] T 86bd546634a948069a9f97673dfc188c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.864785 8151 consensus_queue.cc:1035] T 86bd546634a948069a9f97673dfc188c P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.876600 27062 raft_consensus.cc:1261] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.876965 7221 consensus_queue.cc:1035] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:18.888171 29348 raft_consensus.cc:1261] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 468dd27456c64d67b22e8a58af93d93d [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 01:59:18.888934 7221 consensus_queue.cc:1035] T b98e2f3fb2ea4241a16db8b4bb60b4b8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:18.889597 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:18.892185 26773 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
W20240502 01:59:18.928299 29016 consensus_peers.cc:485] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:18.931578 26930 consensus_queue.cc:237] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:18.932693 26658 raft_consensus.cc:1261] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:18.932757 29348 raft_consensus.cc:1261] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer 842c1b011b5d4c77bfd9b9591111502e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:18.933013 8151 consensus_queue.cc:1035] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:18.933194 7755 consensus_queue.cc:1035] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:18.934840 8151 raft_consensus.cc:2932] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:18.935302 26658 raft_consensus.cc:2932] T 02e988cd07184e97921e7376da20f251 P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:18.935397 29348 raft_consensus.cc:2932] T 02e988cd07184e97921e7376da20f251 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:18.938598 26512 catalog_manager.cc:5484] T 02e988cd07184e97921e7376da20f251 P 842c1b011b5d4c77bfd9b9591111502e reported cstate change: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New cstate: current_term: 1 leader_uuid: "842c1b011b5d4c77bfd9b9591111502e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "842c1b011b5d4c77bfd9b9591111502e" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 40379 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:18.942091 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet 02e988cd07184e97921e7376da20f251 with delete_type TABLET_DATA_TOMBSTONED (TS 0d83fdd1e6e04cf5a5602994d1dbc0df not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:18.942422 8559 tablet_replica.cc:330] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:18.942580 8559 raft_consensus.cc:2229] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:18.942730 8559 raft_consensus.cc:2258] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:18.943343 8559 ts_tablet_manager.cc:1875] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:18.945535 8559 ts_tablet_manager.cc:1888] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:18.945677 8559 log.cc:1196] T 02e988cd07184e97921e7376da20f251 P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/02e988cd07184e97921e7376da20f251
I20240502 01:59:18.946204 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet 02e988cd07184e97921e7376da20f251 (table rebalancer_extra_table_14 [id=4840d0c94b154bc19827ca573fe9d3fe]) successfully deleted
I20240502 01:59:18.955260 27062 raft_consensus.cc:1261] T 9ca8563ff68c4fa590c17ef58e67afa0 P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 01:59:18.955658 8370 consensus_queue.cc:1035] T 9ca8563ff68c4fa590c17ef58e67afa0 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:59:19.071960 29016 consensus_peers.cc:485] T 4b997cb0f369466eacadf8d56b678704 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.074571 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:19.077304 29196 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:19.081594 26773 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:19.082790 26909 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:19.083150 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:19.085651 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:19.091236 29058 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
W20240502 01:59:19.150434 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:19.203886 26998 consensus_peers.cc:485] T dfdd6d487b57479cb48a92bc770947ce P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.251688 26595 consensus_peers.cc:485] T fa9fce27ea7c427aa10d13a7528e793b P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.273306 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:19.274941 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:19.274973 29196 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:19.280500 29058 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 01:59:19.281584 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:19.293431 26773 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:19.295491 26909 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
W20240502 01:59:19.298555 26862 consensus_peers.cc:485] T 67686979ffb54564986fe0733f57a8d2 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.312671 26730 consensus_peers.cc:485] T 052566382186437e8e0245de9f60c30c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.327839 26595 consensus_peers.cc:485] T b546a99821624f0588a4c0f59a5ec5a3 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.364311 26794 consensus_queue.cc:237] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:19.365351 29348 raft_consensus.cc:1261] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:19.365725 8318 consensus_queue.cc:1035] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:19.365896 26658 raft_consensus.cc:1261] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Refusing update from remote peer b290c1bf893444078ed721f25995d124: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 01:59:19.366322 8318 consensus_queue.cc:1035] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 01:59:19.367556 8195 raft_consensus.cc:2932] T e96f6477ca7a497c82c01ef0b2d2747f P b290c1bf893444078ed721f25995d124 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:19.367959 29348 raft_consensus.cc:2932] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:19.371274 26508 catalog_manager.cc:5484] T e96f6477ca7a497c82c01ef0b2d2747f P 468dd27456c64d67b22e8a58af93d93d reported cstate change: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New cstate: current_term: 1 leader_uuid: "b290c1bf893444078ed721f25995d124" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:19.373641 26658 raft_consensus.cc:2932] T e96f6477ca7a497c82c01ef0b2d2747f P 998b57e2badc4d779c586c6579149114 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "b290c1bf893444078ed721f25995d124" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 38007 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:19.373992 29196 tablet_service.cc:1514] Processing DeleteTablet for tablet e96f6477ca7a497c82c01ef0b2d2747f with delete_type TABLET_DATA_TOMBSTONED (TS 0d83fdd1e6e04cf5a5602994d1dbc0df not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:53552
I20240502 01:59:19.374293 8559 tablet_replica.cc:330] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: stopping tablet replica
I20240502 01:59:19.374429 8559 raft_consensus.cc:2229] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 01:59:19.374533 8559 raft_consensus.cc:2258] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 01:59:19.374920 8559 ts_tablet_manager.cc:1875] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 01:59:19.376899 8559 ts_tablet_manager.cc:1888] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 01:59:19.376957 8559 log.cc:1196] T e96f6477ca7a497c82c01ef0b2d2747f P 0d83fdd1e6e04cf5a5602994d1dbc0df: Deleting WAL directory at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/wals/e96f6477ca7a497c82c01ef0b2d2747f
I20240502 01:59:19.377460 26501 catalog_manager.cc:4830] TS 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103): tablet e96f6477ca7a497c82c01ef0b2d2747f (table rebalancer_extra_table_42 [id=6d0ec9e5f1dd4c9e8f0f9a7878fc4f53]) successfully deleted
W20240502 01:59:19.379968 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:19.393412 29016 consensus_peers.cc:485] T b01685bedb5a41bab08b74899e77197c P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.433789 29150 consensus_peers.cc:485] T 273b6d65e3df42ffb072e2129e832a92 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.437819 29079 consensus_queue.cc:237] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:19.439797 29350 raft_consensus.cc:1261] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 01:59:19.440177 29016 consensus_peers.cc:485] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b4f8eed739249cd9fc125d5ef5a6ed8. This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:19.440447 29215 raft_consensus.cc:1261] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Refusing update from remote peer f3d18a019013487ab801599cfb80e4b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:19.440812 8743 consensus_queue.cc:1035] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 01:59:19.441366 8743 consensus_queue.cc:1035] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 01:59:19.442397 29286 consensus_peers.cc:485] T d6e70ac8b41e4127881c5ce5167866de P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.443336 8370 raft_consensus.cc:2932] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:19.443917 29350 raft_consensus.cc:2932] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:19.445226 29215 raft_consensus.cc:2932] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
I20240502 01:59:19.447190 26510 catalog_manager.cc:5484] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4 reported cstate change: config changed from index -1 to 2, VOTER 68e5b1eeb04448fea818a14991eab87c (127.19.43.196) added. New cstate: current_term: 1 leader_uuid: "f3d18a019013487ab801599cfb80e4b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } }
W20240502 01:59:19.458484 26730 consensus_peers.cc:485] T 49ac3a00934944bdb5c9e440045532a3 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.464581 29215 consensus_queue.cc:237] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
W20240502 01:59:19.466058 29286 consensus_peers.cc:485] T c52dc450e0ca4a68af65e924d0e99ce3 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.466300 27064 raft_consensus.cc:1261] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Refusing update from remote peer 0d83fdd1e6e04cf5a5602994d1dbc0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20240502 01:59:19.466455 29149 consensus_peers.cc:485] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb13643f981847c4bb480b74a4764c4c. This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:19.466735 29150 consensus_peers.cc:485] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:19.466966 8272 consensus_queue.cc:1035] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20240502 01:59:19.482906 26862 consensus_peers.cc:485] T 2c150446ded240a2946d5afa10ecd1df P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.483425 29150 consensus_peers.cc:485] T 1c851506649546cb96d13a8c4ea33797 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.493643 29150 consensus_peers.cc:485] T 0831dc2504214ec69f0e5d9556fe6f1e P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.513509 26730 consensus_peers.cc:485] T 9ef794b6e73b4cc9bde557660033114d P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.521795 29016 consensus_peers.cc:485] T 7d48607cfc6c4757ab2df41edb6da47d P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.541056 29150 consensus_peers.cc:485] T 398742b2cb3e41f1bc16fa37a5168809 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.573263 29016 consensus_peers.cc:485] T 4caef235274947e48765e1c11669112a P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.583834 7447 raft_consensus.cc:482] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c693d0933c64d9babc07f404df45cd1)
I20240502 01:59:19.583930 7447 raft_consensus.cc:504] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } }
I20240502 01:59:19.584311 7447 leader_election.cc:288] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259), 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635), f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:19.584967 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e66b19b5551b4011bf1bbb6234a2aba3" candidate_uuid: "468dd27456c64d67b22e8a58af93d93d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "68e5b1eeb04448fea818a14991eab87c" is_pre_election: true
W20240502 01:59:19.585048 29283 leader_election.cc:341] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601): Not found: Tablet not found: e66b19b5551b4011bf1bbb6234a2aba3
I20240502 01:59:19.585093 27064 raft_consensus.cc:2454] T e66b19b5551b4011bf1bbb6234a2aba3 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468dd27456c64d67b22e8a58af93d93d in term 1.
W20240502 01:59:19.585387 29286 leader_election.cc:334] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:19.585448 29286 leader_election.cc:302] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 468dd27456c64d67b22e8a58af93d93d, 68e5b1eeb04448fea818a14991eab87c; no voters: 7c693d0933c64d9babc07f404df45cd1, f3d18a019013487ab801599cfb80e4b4
I20240502 01:59:19.585580 7447 raft_consensus.cc:2726] T e66b19b5551b4011bf1bbb6234a2aba3 P 468dd27456c64d67b22e8a58af93d93d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 01:59:19.592059 29150 consensus_peers.cc:485] T 11a922c5a8994e92a4112d89ce8a9f3a P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.599649 26595 consensus_peers.cc:485] T 734e4d2d6ca746dabc297b73c7744ca8 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.604560 26595 consensus_peers.cc:485] T e1cd4681de8645ff92b479ccdd3eac37 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.636085 29286 consensus_peers.cc:485] T 00d8c5f39ce544018d3f1aa9138e268a P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.636504 29286 consensus_peers.cc:485] T b22135977bb840ed958a414b31489016 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.643574 26595 consensus_peers.cc:485] T c4a7be37454045528f9126385ff48160 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.687734 26595 consensus_peers.cc:485] T c4a41f32520b48d7895e31ca43697f99 P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.688184 26595 consensus_peers.cc:485] T 4c8b72ba18fd4d21bd3c8bb2a1efd18c P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.689888 29016 consensus_peers.cc:485] T 81cd999f76e846bc944a19f7192883be P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.702214 26730 consensus_peers.cc:485] T efa94768c376418396d65a10f5d735c4 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.704633 29286 consensus_peers.cc:485] T a87bdf7578c94ef8b28c5f79227f0af8 P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.716687 26998 consensus_peers.cc:485] T c46a7ee366464769b8498c4208ce8e9b P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.725075 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:19.750211 26862 consensus_peers.cc:485] T 4a528d9bbfa2488e8ce1e7b99b9e4d49 P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.796862 26730 consensus_peers.cc:485] T 7b2aaf8eaa844c0390909ccfc0b4bb09 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.798647 29016 consensus_peers.cc:485] T b1de60ca7a73404abcc530b17bc20662 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.814694 26862 consensus_peers.cc:485] T 626774246b9445bc838427358e08f2cd P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.815507 26730 consensus_peers.cc:485] T d0ffb725f2d843f8a03d8d0b711ce02d P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.815886 26862 consensus_peers.cc:485] T 83db30db2e6f4346822d5d55a1ce1e8f P 842c1b011b5d4c77bfd9b9591111502e -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.816825 29286 consensus_peers.cc:485] T 30eb01a4663b473faa96285b3eb7871a P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.834507 26595 consensus_peers.cc:485] T 213c37d37f1c4a3c8fbc48deafa9edac P 998b57e2badc4d779c586c6579149114 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.858305 8765 ts_tablet_manager.cc:902] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: Initiating tablet copy from peer 0d83fdd1e6e04cf5a5602994d1dbc0df (127.19.43.198:40103)
I20240502 01:59:19.858804 8765 tablet_copy_client.cc:308] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.198:40103
I20240502 01:59:19.859180 29221 tablet_copy_service.cc:140] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Received BeginTabletCopySession request for tablet bb13643f981847c4bb480b74a4764c4c from peer 998b57e2badc4d779c586c6579149114 ({username='slave'} at 127.19.43.193:44399)
I20240502 01:59:19.859264 29221 tablet_copy_service.cc:161] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Beginning new tablet copy session on tablet bb13643f981847c4bb480b74a4764c4c from peer 998b57e2badc4d779c586c6579149114 at {username='slave'} at 127.19.43.193:44399: session id = 998b57e2badc4d779c586c6579149114-bb13643f981847c4bb480b74a4764c4c
I20240502 01:59:19.860049 29221 tablet_copy_source_session.cc:205] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:19.860930 8765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb13643f981847c4bb480b74a4764c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:19.862897 8765 tablet_copy_client.cc:783] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:19.863054 8765 tablet_copy_client.cc:647] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:19.864042 8765 tablet_copy_client.cc:523] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:19.865053 8765 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: Bootstrap starting.
W20240502 01:59:19.868052 29286 consensus_peers.cc:485] T 1a9927f6676840dea245baa652c58b7d P 468dd27456c64d67b22e8a58af93d93d -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:19.869165 8765 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 01:59:19.869488 8765 tablet_bootstrap.cc:492] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: Bootstrap complete.
I20240502 01:59:19.869652 8765 ts_tablet_manager.cc:1367] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20240502 01:59:19.870138 8765 raft_consensus.cc:348] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114 [term 2 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } }
I20240502 01:59:19.870374 8765 raft_consensus.cc:729] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998b57e2badc4d779c586c6579149114, State: Initialized, Role: FOLLOWER
I20240502 01:59:19.870483 8765 consensus_queue.cc:260] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } }
I20240502 01:59:19.870851 8765 ts_tablet_manager.cc:1398] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 01:59:19.871843 29221 tablet_copy_service.cc:342] P 0d83fdd1e6e04cf5a5602994d1dbc0df: Request end of tablet copy session 998b57e2badc4d779c586c6579149114-bb13643f981847c4bb480b74a4764c4c received from {username='slave'} at 127.19.43.193:44399
I20240502 01:59:19.871918 29221 tablet_copy_service.cc:433] P 0d83fdd1e6e04cf5a5602994d1dbc0df: ending tablet copy session 998b57e2badc4d779c586c6579149114-bb13643f981847c4bb480b74a4764c4c on tablet bb13643f981847c4bb480b74a4764c4c with peer 998b57e2badc4d779c586c6579149114
I20240502 01:59:19.893630 8771 ts_tablet_manager.cc:902] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: Initiating tablet copy from peer f3d18a019013487ab801599cfb80e4b4 (127.19.43.197:34601)
I20240502 01:59:19.894063 8771 tablet_copy_client.cc:308] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Beginning tablet copy session from remote peer at address 127.19.43.197:34601
I20240502 01:59:19.894466 29102 tablet_copy_service.cc:140] P f3d18a019013487ab801599cfb80e4b4: Received BeginTabletCopySession request for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8 from peer 68e5b1eeb04448fea818a14991eab87c ({username='slave'} at 127.19.43.196:54885)
I20240502 01:59:19.894563 29102 tablet_copy_service.cc:161] P f3d18a019013487ab801599cfb80e4b4: Beginning new tablet copy session on tablet 1b4f8eed739249cd9fc125d5ef5a6ed8 from peer 68e5b1eeb04448fea818a14991eab87c at {username='slave'} at 127.19.43.196:54885: session id = 68e5b1eeb04448fea818a14991eab87c-1b4f8eed739249cd9fc125d5ef5a6ed8
I20240502 01:59:19.895243 29102 tablet_copy_source_session.cc:205] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P f3d18a019013487ab801599cfb80e4b4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 01:59:19.895977 8771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b4f8eed739249cd9fc125d5ef5a6ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:19.898000 8771 tablet_copy_client.cc:783] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 0 data blocks...
I20240502 01:59:19.898164 8771 tablet_copy_client.cc:647] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Starting download of 1 WAL segments...
I20240502 01:59:19.899135 8771 tablet_copy_client.cc:523] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 01:59:19.900310 8771 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap starting.
I20240502 01:59:19.902988 8771 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 01:59:19.903177 8771 tablet_bootstrap.cc:492] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: Bootstrap complete.
I20240502 01:59:19.903321 8771 ts_tablet_manager.cc:1367] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240502 01:59:19.903713 8771 raft_consensus.cc:348] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:19.903815 8771 raft_consensus.cc:729] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e5b1eeb04448fea818a14991eab87c, State: Initialized, Role: FOLLOWER
I20240502 01:59:19.903930 8771 consensus_queue.cc:260] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "468dd27456c64d67b22e8a58af93d93d" member_type: VOTER last_known_addr { host: "127.19.43.199" port: 44303 } } peers { permanent_uuid: "f3d18a019013487ab801599cfb80e4b4" member_type: VOTER last_known_addr { host: "127.19.43.197" port: 34601 } } peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } }
I20240502 01:59:19.904181 8771 ts_tablet_manager.cc:1398] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:19.904560 29102 tablet_copy_service.cc:342] P f3d18a019013487ab801599cfb80e4b4: Request end of tablet copy session 68e5b1eeb04448fea818a14991eab87c-1b4f8eed739249cd9fc125d5ef5a6ed8 received from {username='slave'} at 127.19.43.196:54885
I20240502 01:59:19.904636 29102 tablet_copy_service.cc:433] P f3d18a019013487ab801599cfb80e4b4: ending tablet copy session 68e5b1eeb04448fea818a14991eab87c-1b4f8eed739249cd9fc125d5ef5a6ed8 on tablet 1b4f8eed739249cd9fc125d5ef5a6ed8 with peer 68e5b1eeb04448fea818a14991eab87c
W20240502 01:59:19.923501 29016 consensus_peers.cc:485] T 714bb06a8a56413f8210f86c1fa9f717 P f3d18a019013487ab801599cfb80e4b4 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:19.936586 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
W20240502 01:59:19.997581 26998 consensus_peers.cc:485] T 7ca17309584740f28f6a9163a2c2198c P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:20.010777 26998 consensus_peers.cc:485] T cbeef9995d6e46dfbbff976d0bb9028e P 68e5b1eeb04448fea818a14991eab87c -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 01:59:20.322244 26730 consensus_peers.cc:462] T 565aab51035c4f328e49eca53eddf621 P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:20.344435 26658 raft_consensus.cc:1203] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20240502 01:59:20.345016 8272 raft_consensus.cc:2932] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:20.345369 26658 raft_consensus.cc:2932] T bb13643f981847c4bb480b74a4764c4c P 998b57e2badc4d779c586c6579149114 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:20.349115 27064 raft_consensus.cc:2932] T bb13643f981847c4bb480b74a4764c4c P 68e5b1eeb04448fea818a14991eab87c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:20.349921 26508 catalog_manager.cc:5484] T bb13643f981847c4bb480b74a4764c4c P 0d83fdd1e6e04cf5a5602994d1dbc0df reported cstate change: config changed from index -1 to 3, VOTER 998b57e2badc4d779c586c6579149114 (127.19.43.193) added. New cstate: current_term: 2 leader_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0d83fdd1e6e04cf5a5602994d1dbc0df" member_type: VOTER last_known_addr { host: "127.19.43.198" port: 40103 } } peers { permanent_uuid: "68e5b1eeb04448fea818a14991eab87c" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 43635 } } peers { permanent_uuid: "7c693d0933c64d9babc07f404df45cd1" member_type: VOTER last_known_addr { host: "127.19.43.200" port: 36259 } } peers { permanent_uuid: "998b57e2badc4d779c586c6579149114" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 32929 } } }
I20240502 01:59:20.371088 27064 raft_consensus.cc:1203] T 1b4f8eed739249cd9fc125d5ef5a6ed8 P 68e5b1eeb04448fea818a14991eab87c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 01:59:20.453990 26730 consensus_peers.cc:462] T dc99f8bfb06c4bcda6111b218125826c P b290c1bf893444078ed721f25995d124 -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:20.525233 29196 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:20.525383 29328 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 01:59:20.525638 27043 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 01:59:20.534209 26641 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 01:59:20.540501 26909 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 01:59:20.549347 29058 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 01:59:20.564257 26773 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/rebalancer_tool-test.cc:1203: Failure
Value of: s.ok()
Actual: false
Expected: true
Runtime error: /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu: process exited with non-zero status 1
stdout: Per-server replica distribution summary:
Statistic | Value
-----------------------+-----------
Minimum Replica Count | 0
Maximum Replica Count | 46
Average Replica Count | 22.875000
Per-table replica distribution summary:
Replica Skew | Value
--------------+----------
Minimum | 1
Maximum | 3
Average | 2.904762
stderr: W20240502 01:58:46.029698 29547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:58:46.029891 29547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:58:46.032770 29547 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 01:58:46.459244 29547 rebalancer_tool.cc:215] running whole-cluster rebalancing
W20240502 01:58:46.736385 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 682bab6cacdb4110a2125888f67c7c84: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.736469 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 3a48f5c202c04fe8bc5c1901bd327d46: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.736634 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d591439fcb3647eea2d27e598437f007: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:46.736663 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 911a1465522146b09f03a60b31759670: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.736693 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d7e58e10bc3742c694eee2f14db77982: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:46.736721 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 1289082436504a27bf6970f9b81fb4ef: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.736750 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8bd0d307cf9b4c90b9f1bc7cc8fdeb01: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.736779 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 24546c62f4d34ec5a528011afd24c010: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.736804 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 682bab6cacdb4110a2125888f67c7c84: could not find any suitable replica to move from server 468dd27456c64d67b22e8a58af93d93d to server b290c1bf893444078ed721f25995d124
W20240502 01:58:46.736912 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 60a3751a3295452a8c5c748f9b0c81af: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.736939 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d341bb6a432647ab90b4207f29bfa6f2: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.736968 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8417deae11f14e4e8e5d8d5fddd562ed: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.736994 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 68ae40c8c303404c8908467f3134217f: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.737020 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5d5338285c53470eb5116a351534a54a: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.737093 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8bd0d307cf9b4c90b9f1bc7cc8fdeb01: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.737133 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 74ba0b4ded494b7da3c1f15a957044ad: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.737174 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ba4c83ebc427424aa2e600c40f538972: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:46.737202 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d1cbab12dd3d44308b638d046df39658: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:46.737228 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table fe6318fa9234475b94b8597d4f9fe9d1: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.737255 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table e8b7140ce14c490fa251594750e1da52: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.737282 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ad62d3bfb6624574ba0b8907ab3885bb: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.737314 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cdfbad05a8f14dc6bceb10ffc9f00c82: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.737351 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4c7df5b205f148308c18bd697c83e7fb: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.737380 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d341bb6a432647ab90b4207f29bfa6f2: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:46.737403 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8417deae11f14e4e8e5d8d5fddd562ed: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.737435 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5d5338285c53470eb5116a351534a54a: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:46.737462 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 68ae40c8c303404c8908467f3134217f: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:46.737493 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d591439fcb3647eea2d27e598437f007: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:46.737524 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 911a1465522146b09f03a60b31759670: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:46.737555 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 74ba0b4ded494b7da3c1f15a957044ad: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 7c693d0933c64d9babc07f404df45cd1
I20240502 01:58:46.768915 29547 rebalancer_tool.cc:1074] tablet b5439ff11ca04925bf46e736a5a53487: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:46.792917 29547 rebalancer_tool.cc:1074] tablet b265eec55ee1406b9ff99c159304fa65: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:46.829646 29547 rebalancer_tool.cc:1074] tablet 565aab51035c4f328e49eca53eddf621: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:46.852970 29547 rebalancer_tool.cc:1074] tablet cceb8a91997e4db9a4842df26ad3b02e: 'b290c1bf893444078ed721f25995d124' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:46.878582 29547 rebalancer_tool.cc:1074] tablet 6e2ebe00a52240cb9ac5fc8fea9d7151: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:46.900020 29547 rebalancer_tool.cc:1074] tablet 1c002ce2d74547399685b69a0a4cbada: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:46.943400 29547 rebalancer_tool.cc:1074] tablet 936e77ced55c41d6bfe955d9a4271b80: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:46.973174 29547 rebalancer_tool.cc:1074] tablet 7f5513cf299e44038525ad5f438d74ec: 'b290c1bf893444078ed721f25995d124' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:46.997054 29547 rebalancer_tool.cc:1074] tablet e812884ef6474b5384d026637dc62b61: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:47.020274 29547 rebalancer_tool.cc:1074] tablet cdd32c8294b44540ac5b832cc48d2519: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:47.053314 29547 rebalancer_tool.cc:1074] tablet ce3d632b976e491e8ac726952c69cbc3: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:47.080621 29547 rebalancer_tool.cc:1074] tablet c187cf1abe124e73badcb06ca1a489c4: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:47.105808 29547 rebalancer_tool.cc:1074] tablet 6e9b0960a2a94c3fb90902c126ccd034: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:47.129480 29547 rebalancer_tool.cc:1074] tablet 039270108deb4f55b8458ceccf31daa6: '68e5b1eeb04448fea818a14991eab87c' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:47.158341 29547 rebalancer_tool.cc:1074] tablet d3e2d17af0194d5d8c08ad8372b1e9c1: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:47.208958 29547 rebalancer_tool.cc:1074] tablet ae9f78db63b74beca267ff5f12a211ae: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:47.240350 29547 rebalancer_tool.cc:1074] tablet 1417ea39bb2b4716bfb3d11c05853c03: '998b57e2badc4d779c586c6579149114' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:47.265909 29547 rebalancer_tool.cc:1074] tablet e1cd4681de8645ff92b479ccdd3eac37: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:47.304957 29547 rebalancer_tool.cc:1074] tablet e60816eacfb547cda5a6c87c68bf101d: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:47.330472 29547 rebalancer_tool.cc:1074] tablet 507cc922d2884f48b0f4c43185b4225d: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:49.405004 29547 rebalancer_tool.cc:1123] tablet 1c002ce2d74547399685b69a0a4cbada: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move is abandoned: Not found: 1c002ce2d74547399685b69a0a4cbada: tablet not found
E20240502 01:58:49.448132 30970 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table_new_name: Not found: the table does not exist: table_name: "rebalancer_dynamic_table_new_name"
I20240502 01:58:50.069939 29547 rebalancer_tool.cc:1123] tablet b5439ff11ca04925bf46e736a5a53487: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move is abandoned: Not found: b5439ff11ca04925bf46e736a5a53487: tablet not found
I20240502 01:58:50.549916 29547 rebalancer_tool.cc:1123] tablet ce3d632b976e491e8ac726952c69cbc3: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move is abandoned: Not found: ce3d632b976e491e8ac726952c69cbc3: tablet not found
I20240502 01:58:50.558678 29547 rebalancer_tool.cc:1123] tablet e812884ef6474b5384d026637dc62b61: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move is abandoned: Not found: e812884ef6474b5384d026637dc62b61: tablet not found
I20240502 01:58:51.106086 29547 rebalancer_tool.cc:756] re-synchronizing cluster state
W20240502 01:58:51.361243 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8417deae11f14e4e8e5d8d5fddd562ed: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:51.361325 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 3a48f5c202c04fe8bc5c1901bd327d46: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:51.361493 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 24546c62f4d34ec5a528011afd24c010: could not find any suitable replica to move from server 842c1b011b5d4c77bfd9b9591111502e to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:51.361550 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 3a48f5c202c04fe8bc5c1901bd327d46: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server f3d18a019013487ab801599cfb80e4b4
W20240502 01:58:51.361600 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 68ae40c8c303404c8908467f3134217f: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:51.361647 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cdfbad05a8f14dc6bceb10ffc9f00c82: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:51.361675 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ad62d3bfb6624574ba0b8907ab3885bb: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:51.361713 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 911a1465522146b09f03a60b31759670: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:51.361747 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d7e58e10bc3742c694eee2f14db77982: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
W20240502 01:58:51.361785 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 60a3751a3295452a8c5c748f9b0c81af: could not find any suitable replica to move from server 998b57e2badc4d779c586c6579149114 to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:58:51.361814 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5942d660595e4276b446bb801e4ba849: could not find any suitable replica to move from server 0d83fdd1e6e04cf5a5602994d1dbc0df to server 68e5b1eeb04448fea818a14991eab87c
W20240502 01:58:51.361851 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table e8b7140ce14c490fa251594750e1da52: could not find any suitable replica to move from server b290c1bf893444078ed721f25995d124 to server 7c693d0933c64d9babc07f404df45cd1
W20240502 01:58:51.361883 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8417deae11f14e4e8e5d8d5fddd562ed: could not find any suitable replica to move from server 68e5b1eeb04448fea818a14991eab87c to server b290c1bf893444078ed721f25995d124
W20240502 01:58:51.361907 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table fbfb5e4da94848e38da0799868b0d490: could not find any suitable replica to move from server 7c693d0933c64d9babc07f404df45cd1 to server 0d83fdd1e6e04cf5a5602994d1dbc0df
I20240502 01:58:51.387997 29547 rebalancer_tool.cc:1074] tablet c455f182f874431a88a46a8aff2b1675: '7c693d0933c64d9babc07f404df45cd1' -> '998b57e2badc4d779c586c6579149114' move scheduled
I20240502 01:58:51.427589 29547 rebalancer_tool.cc:1074] tablet dc99f8bfb06c4bcda6111b218125826c: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:51.455200 29547 rebalancer_tool.cc:1074] tablet 50bfc86904bc4347bd6beb7f16eb638d: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:51.482522 29547 rebalancer_tool.cc:1074] tablet dfdd6d487b57479cb48a92bc770947ce: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:51.777602 29547 rebalancer_tool.cc:1123] tablet 507cc922d2884f48b0f4c43185b4225d: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move is abandoned: Not found: 507cc922d2884f48b0f4c43185b4225d: tablet not found
I20240502 01:58:51.778678 29547 rebalancer_tool.cc:1123] tablet 1417ea39bb2b4716bfb3d11c05853c03: '998b57e2badc4d779c586c6579149114' -> '468dd27456c64d67b22e8a58af93d93d' move is abandoned: Not found: 1417ea39bb2b4716bfb3d11c05853c03: tablet not found
I20240502 01:58:51.779346 29547 rebalancer_tool.cc:1123] tablet 6e2ebe00a52240cb9ac5fc8fea9d7151: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move is abandoned: Not found: 6e2ebe00a52240cb9ac5fc8fea9d7151: tablet not found
I20240502 01:58:51.784734 29547 rebalancer_tool.cc:1136] tablet 936e77ced55c41d6bfe955d9a4271b80: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:51.785480 29547 rebalancer_tool.cc:1123] tablet e60816eacfb547cda5a6c87c68bf101d: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move is abandoned: Not found: e60816eacfb547cda5a6c87c68bf101d: tablet not found
I20240502 01:58:51.790913 29547 rebalancer_tool.cc:1136] tablet 7f5513cf299e44038525ad5f438d74ec: 'b290c1bf893444078ed721f25995d124' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:58:52.210170 29547 rebalancer_tool.cc:1136] tablet cceb8a91997e4db9a4842df26ad3b02e: 'b290c1bf893444078ed721f25995d124' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:58:52.210920 29547 rebalancer_tool.cc:1123] tablet 6e9b0960a2a94c3fb90902c126ccd034: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move is abandoned: Not found: 6e9b0960a2a94c3fb90902c126ccd034: tablet not found
I20240502 01:58:52.216091 29547 rebalancer_tool.cc:1136] tablet e1cd4681de8645ff92b479ccdd3eac37: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:58:52.220911 29547 rebalancer_tool.cc:1136] tablet 565aab51035c4f328e49eca53eddf621: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:52.221581 29547 rebalancer_tool.cc:1123] tablet d3e2d17af0194d5d8c08ad8372b1e9c1: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move is abandoned: Not found: d3e2d17af0194d5d8c08ad8372b1e9c1: tablet not found
I20240502 01:58:52.226260 29547 rebalancer_tool.cc:1136] tablet b265eec55ee1406b9ff99c159304fa65: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:58:52.231424 29547 rebalancer_tool.cc:1136] tablet cdd32c8294b44540ac5b832cc48d2519: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:58:52.236236 29547 rebalancer_tool.cc:1136] tablet c187cf1abe124e73badcb06ca1a489c4: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:52.241287 29547 rebalancer_tool.cc:1136] tablet 039270108deb4f55b8458ceccf31daa6: '68e5b1eeb04448fea818a14991eab87c' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:58:52.245769 29547 rebalancer_tool.cc:1136] tablet ae9f78db63b74beca267ff5f12a211ae: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:52.268882 29547 rebalancer_tool.cc:1074] tablet 6acba26a44294eb284d03176631ae39b: '68e5b1eeb04448fea818a14991eab87c' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:58:52.295722 29547 rebalancer_tool.cc:1074] tablet 99da1c4053674589b87a7898ed9d57de: '68e5b1eeb04448fea818a14991eab87c' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:52.331563 29547 rebalancer_tool.cc:1074] tablet ee632f93fe234584814e6aa7b6a63bc0: '468dd27456c64d67b22e8a58af93d93d' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
I20240502 01:58:52.357018 29547 rebalancer_tool.cc:1074] tablet e1950ace91824415b9bf3d1657bad72f: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:52.381889 29547 rebalancer_tool.cc:1074] tablet 8e66776cedd8407dad9c187b0b04f073: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '68e5b1eeb04448fea818a14991eab87c' move scheduled
I20240502 01:58:52.408373 29547 rebalancer_tool.cc:1074] tablet 8afd4c5362464c4cba43d5ef28b50c29: '7c693d0933c64d9babc07f404df45cd1' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:52.434338 29547 rebalancer_tool.cc:1074] tablet e24e0bd4a1df4fed876e1c4c50af7d70: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:58:52.461185 29547 rebalancer_tool.cc:1074] tablet 2e129aa347444eafab51845d9bf2ae32: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:52.488109 29547 rebalancer_tool.cc:1074] tablet 5515469fdf494900ae20f20fdfca587c: '68e5b1eeb04448fea818a14991eab87c' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
W20240502 01:58:52.547565 32313 meta_cache.cc:1015] Not found: LookupRpc { table: 'rebalancer_dynamic_table_new_name', partition-key: (<start>), attempt: 1 } failed: table rebalancer_dynamic_table_new_name was deleted: Table deleted at 2024-05-02 01:58:52 UTC
I20240502 01:58:56.500519 29547 rebalancer_tool.cc:1136] tablet e1950ace91824415b9bf3d1657bad72f: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:56.513293 29547 rebalancer_tool.cc:1136] tablet 2e129aa347444eafab51845d9bf2ae32: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:58:56.520555 29547 rebalancer_tool.cc:1136] tablet ee632f93fe234584814e6aa7b6a63bc0: '468dd27456c64d67b22e8a58af93d93d' -> '842c1b011b5d4c77bfd9b9591111502e' move completed: OK
I20240502 01:58:56.525578 29547 rebalancer_tool.cc:1136] tablet 99da1c4053674589b87a7898ed9d57de: '68e5b1eeb04448fea818a14991eab87c' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:56.530776 29547 rebalancer_tool.cc:1136] tablet 6acba26a44294eb284d03176631ae39b: '68e5b1eeb04448fea818a14991eab87c' -> 'b290c1bf893444078ed721f25995d124' move completed: OK
I20240502 01:58:56.535964 29547 rebalancer_tool.cc:1136] tablet 8afd4c5362464c4cba43d5ef28b50c29: '7c693d0933c64d9babc07f404df45cd1' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:58:56.541411 29547 rebalancer_tool.cc:1136] tablet 50bfc86904bc4347bd6beb7f16eb638d: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:58:56.546450 29547 rebalancer_tool.cc:1136] tablet c455f182f874431a88a46a8aff2b1675: '7c693d0933c64d9babc07f404df45cd1' -> '998b57e2badc4d779c586c6579149114' move completed: OK
I20240502 01:58:56.551764 29547 rebalancer_tool.cc:1136] tablet dfdd6d487b57479cb48a92bc770947ce: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:58:56.556972 29547 rebalancer_tool.cc:1136] tablet dc99f8bfb06c4bcda6111b218125826c: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:58:56.593046 29547 rebalancer_tool.cc:1074] tablet 47f84b6ffb074bc682aa7b087b8c3614: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:56.622139 29547 rebalancer_tool.cc:1074] tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:56.649479 29547 rebalancer_tool.cc:1074] tablet a0ea54806f7342e2bb3d2f287bb84bdc: '998b57e2badc4d779c586c6579149114' -> '68e5b1eeb04448fea818a14991eab87c' move scheduled
I20240502 01:58:56.672716 29547 rebalancer_tool.cc:1074] tablet 3d53d3130fa84270ad934353a5dbab41: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:56.695978 29547 rebalancer_tool.cc:1074] tablet a326a7899c064c95b426aae61d65208f: 'f3d18a019013487ab801599cfb80e4b4' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:58:56.728260 29547 rebalancer_tool.cc:1074] tablet c46a7ee366464769b8498c4208ce8e9b: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:56.754447 29547 rebalancer_tool.cc:1074] tablet 626774246b9445bc838427358e08f2cd: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:58:56.780781 29547 rebalancer_tool.cc:1074] tablet a0d96b487a964b90b720d8977a96fc5b: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:58:56.803349 29547 rebalancer_tool.cc:1074] tablet 2c150446ded240a2946d5afa10ecd1df: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:58:58.533812 29547 rebalancer_tool.cc:1136] tablet 5515469fdf494900ae20f20fdfca587c: '68e5b1eeb04448fea818a14991eab87c' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:58.776697 29547 rebalancer_tool.cc:1136] tablet e24e0bd4a1df4fed876e1c4c50af7d70: 'b290c1bf893444078ed721f25995d124' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:58:59.107421 29547 rebalancer_tool.cc:1136] tablet 8e66776cedd8407dad9c187b0b04f073: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '68e5b1eeb04448fea818a14991eab87c' move completed: OK
I20240502 01:58:59.132524 29547 rebalancer_tool.cc:1074] tablet d14b1389d460401b83d3652b25aaddec: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:59:00.176071 29547 rebalancer_tool.cc:1136] tablet c46a7ee366464769b8498c4208ce8e9b: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:00.633741 29547 rebalancer_tool.cc:1136] tablet 3d53d3130fa84270ad934353a5dbab41: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:59:00.640895 29547 rebalancer_tool.cc:1136] tablet a0ea54806f7342e2bb3d2f287bb84bdc: '998b57e2badc4d779c586c6579149114' -> '68e5b1eeb04448fea818a14991eab87c' move completed: OK
I20240502 01:59:00.646111 29547 rebalancer_tool.cc:1136] tablet 47f84b6ffb074bc682aa7b087b8c3614: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:59:00.651840 29547 rebalancer_tool.cc:1136] tablet 4a528d9bbfa2488e8ce1e7b99b9e4d49: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:00.677829 29547 rebalancer_tool.cc:1074] tablet 7b2aaf8eaa844c0390909ccfc0b4bb09: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:00.704753 29547 rebalancer_tool.cc:1074] tablet bde4108189cc45a5a1e8ae2d0ef699fd: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:59:00.737324 29547 rebalancer_tool.cc:1074] tablet 49ac3a00934944bdb5c9e440045532a3: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:00.764168 29547 rebalancer_tool.cc:1074] tablet 01764f37959e4c6799e3a83f0d4ed2e1: '998b57e2badc4d779c586c6579149114' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
I20240502 01:59:01.611135 29547 rebalancer_tool.cc:1136] tablet 2c150446ded240a2946d5afa10ecd1df: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:01.620175 29547 rebalancer_tool.cc:1136] tablet a0d96b487a964b90b720d8977a96fc5b: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
I20240502 01:59:01.875121 29547 rebalancer_tool.cc:1136] tablet 626774246b9445bc838427358e08f2cd: '68e5b1eeb04448fea818a14991eab87c' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:01.880985 29547 rebalancer_tool.cc:1136] tablet a326a7899c064c95b426aae61d65208f: 'f3d18a019013487ab801599cfb80e4b4' -> 'b290c1bf893444078ed721f25995d124' move completed: OK
I20240502 01:59:01.907474 29547 rebalancer_tool.cc:1074] tablet 052566382186437e8e0245de9f60c30c: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:01.949839 29547 rebalancer_tool.cc:1074] tablet 69df2cdb31d640ff9f43bb395f2f26fe: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
W20240502 01:59:02.413089 3448 meta_cache.cc:1015] Not found: LookupRpc { table: 'rebalancer_dynamic_table_new_name', partition-key: (<start>), attempt: 1 } failed: table rebalancer_dynamic_table_new_name was deleted: Table deleted at 2024-05-02 01:59:02 UTC
I20240502 01:59:03.089912 29547 rebalancer_tool.cc:1136] tablet d14b1389d460401b83d3652b25aaddec: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> 'b290c1bf893444078ed721f25995d124' move completed: OK
I20240502 01:59:03.101410 29547 rebalancer_tool.cc:1136] tablet bde4108189cc45a5a1e8ae2d0ef699fd: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:59:03.129953 29547 rebalancer_tool.cc:1074] tablet dcf3a090ea284d12979b161f59081ff7: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:59:03.951982 29547 rebalancer_tool.cc:1136] tablet 01764f37959e4c6799e3a83f0d4ed2e1: '998b57e2badc4d779c586c6579149114' -> '842c1b011b5d4c77bfd9b9591111502e' move completed: OK
I20240502 01:59:03.968420 29547 rebalancer_tool.cc:1136] tablet 49ac3a00934944bdb5c9e440045532a3: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:03.974342 29547 rebalancer_tool.cc:1136] tablet 7b2aaf8eaa844c0390909ccfc0b4bb09: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:04.003841 29547 rebalancer_tool.cc:1074] tablet c2563da5db7a4e5197532b31e9aef636: '7c693d0933c64d9babc07f404df45cd1' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
W20240502 01:59:04.080014 4046 meta_cache.cc:1015] Not found: LookupRpc { table: 'rebalancer_dynamic_table_new_name', partition-key: (<start>), attempt: 1 } failed: table rebalancer_dynamic_table_new_name was deleted: Table deleted at 2024-05-02 01:59:04 UTC
I20240502 01:59:04.453213 29547 rebalancer_tool.cc:1136] tablet 69df2cdb31d640ff9f43bb395f2f26fe: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:59:04.458119 29547 rebalancer_tool.cc:1136] tablet 052566382186437e8e0245de9f60c30c: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:04.694816 29547 rebalancer_tool.cc:1136] tablet dcf3a090ea284d12979b161f59081ff7: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:05.677086 29547 rebalancer_tool.cc:1136] tablet c2563da5db7a4e5197532b31e9aef636: '7c693d0933c64d9babc07f404df45cd1' -> '842c1b011b5d4c77bfd9b9591111502e' move completed: OK
W20240502 01:59:05.936456 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0c9a4b7f62d147cdb650dbf3ec335502: could not find any suitable replica to move from server 7c693d0933c64d9babc07f404df45cd1 to server 842c1b011b5d4c77bfd9b9591111502e
W20240502 01:59:05.936937 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0c9a4b7f62d147cdb650dbf3ec335502: could not find any suitable replica to move from server 7c693d0933c64d9babc07f404df45cd1 to server 468dd27456c64d67b22e8a58af93d93d
W20240502 01:59:05.936987 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 6d883fc7f7434cc68481072549e1fb6b: could not find any suitable replica to move from server 7c693d0933c64d9babc07f404df45cd1 to server 998b57e2badc4d779c586c6579149114
W20240502 01:59:05.937065 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table eb586a77305a4629abb1ab153d16db2b: could not find any suitable replica to move from server 7c693d0933c64d9babc07f404df45cd1 to server 998b57e2badc4d779c586c6579149114
W20240502 01:59:05.937109 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table cb4c07c573ee44f4bba16eb6623444ed: could not find any suitable replica to move from server 468dd27456c64d67b22e8a58af93d93d to server b290c1bf893444078ed721f25995d124
W20240502 01:59:05.937160 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ca9eb6b784dd44329e3443cd4a725220: could not find any suitable replica to move from server 0d83fdd1e6e04cf5a5602994d1dbc0df to server b290c1bf893444078ed721f25995d124
W20240502 01:59:05.937223 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4840d0c94b154bc19827ca573fe9d3fe: could not find any suitable replica to move from server 468dd27456c64d67b22e8a58af93d93d to server 68e5b1eeb04448fea818a14991eab87c
W20240502 01:59:05.937474 29547 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d35b9b3b5c44483f9111e1bbe6f89415: could not find any suitable replica to move from server 0d83fdd1e6e04cf5a5602994d1dbc0df to server 842c1b011b5d4c77bfd9b9591111502e
I20240502 01:59:05.964411 29547 rebalancer_tool.cc:1074] tablet 499d2ca7bffc42748591db5280f829f2: '468dd27456c64d67b22e8a58af93d93d' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:59:06.014520 29547 rebalancer_tool.cc:1074] tablet 089d9dc33a594712ac04ad28a0cd37b1: '7c693d0933c64d9babc07f404df45cd1' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
I20240502 01:59:06.040163 29547 rebalancer_tool.cc:1074] tablet 78609c45dc0c4e1b838bb737ebf7e034: '68e5b1eeb04448fea818a14991eab87c' -> '998b57e2badc4d779c586c6579149114' move scheduled
I20240502 01:59:06.065663 29547 rebalancer_tool.cc:1074] tablet 8afd4c5362464c4cba43d5ef28b50c29: 'f3d18a019013487ab801599cfb80e4b4' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:59:06.092020 29547 rebalancer_tool.cc:1074] tablet 273b6d65e3df42ffb072e2129e832a92: '468dd27456c64d67b22e8a58af93d93d' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:59:06.121402 29547 rebalancer_tool.cc:1074] tablet 8209cefcc49042b18fdd97d845d167e2: '7c693d0933c64d9babc07f404df45cd1' -> '68e5b1eeb04448fea818a14991eab87c' move scheduled
I20240502 01:59:06.150530 29547 rebalancer_tool.cc:1074] tablet e1cd4681de8645ff92b479ccdd3eac37: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:59:06.179208 29547 rebalancer_tool.cc:1074] tablet 5515469fdf494900ae20f20fdfca587c: '998b57e2badc4d779c586c6579149114' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:59:06.202800 29547 rebalancer_tool.cc:1074] tablet ab7a9b176213400f990c4d5df0d7567f: '468dd27456c64d67b22e8a58af93d93d' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:59:06.242606 29547 rebalancer_tool.cc:1074] tablet 213c37d37f1c4a3c8fbc48deafa9edac: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:06.272884 29547 rebalancer_tool.cc:1074] tablet d3d8471cf7654dbdab51d855ed30be18: 'f3d18a019013487ab801599cfb80e4b4' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
I20240502 01:59:06.299919 29547 rebalancer_tool.cc:1074] tablet a0d96b487a964b90b720d8977a96fc5b: '998b57e2badc4d779c586c6579149114' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:59:06.359795 29547 rebalancer_tool.cc:1074] tablet 6a5f8bfeddd1443da789f67949830f99: 'b290c1bf893444078ed721f25995d124' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:59:06.400954 29547 rebalancer_tool.cc:1074] tablet b546a99821624f0588a4c0f59a5ec5a3: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:06.429970 29547 rebalancer_tool.cc:1074] tablet 7f701a3437f84a84b6e3290266d24c94: 'f3d18a019013487ab801599cfb80e4b4' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
I20240502 01:59:06.465201 29547 rebalancer_tool.cc:1074] tablet d216b8ccd8ea43df85c1013cdaf36564: '998b57e2badc4d779c586c6579149114' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
I20240502 01:59:06.489789 29547 rebalancer_tool.cc:1074] tablet 626774246b9445bc838427358e08f2cd: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:06.517621 29547 rebalancer_tool.cc:1074] tablet 9ca8563ff68c4fa590c17ef58e67afa0: '468dd27456c64d67b22e8a58af93d93d' -> '68e5b1eeb04448fea818a14991eab87c' move scheduled
I20240502 01:59:06.544790 29547 rebalancer_tool.cc:1074] tablet e1950ace91824415b9bf3d1657bad72f: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move scheduled
E20240502 01:59:12.383884 6596 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table_new_name: Not found: the table does not exist: table_name: "rebalancer_dynamic_table_new_name"
I20240502 01:59:14.661593 29547 rebalancer_tool.cc:1136] tablet 6a5f8bfeddd1443da789f67949830f99: 'b290c1bf893444078ed721f25995d124' -> '468dd27456c64d67b22e8a58af93d93d' move completed: OK
W20240502 01:59:14.776494 7176 meta_cache.cc:1015] Not found: LookupRpc { table: 'rebalancer_dynamic_table_new_name', partition-key: (<start>), attempt: 1 } failed: table rebalancer_dynamic_table_new_name was deleted: Table deleted at 2024-05-02 01:59:14 UTC
I20240502 01:59:14.970993 29547 rebalancer_tool.cc:1136] tablet a0d96b487a964b90b720d8977a96fc5b: '998b57e2badc4d779c586c6579149114' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:14.977077 29547 rebalancer_tool.cc:1136] tablet d3d8471cf7654dbdab51d855ed30be18: 'f3d18a019013487ab801599cfb80e4b4' -> '842c1b011b5d4c77bfd9b9591111502e' move completed: OK
I20240502 01:59:14.982831 29547 rebalancer_tool.cc:1136] tablet 213c37d37f1c4a3c8fbc48deafa9edac: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:14.988487 29547 rebalancer_tool.cc:1136] tablet ab7a9b176213400f990c4d5df0d7567f: '468dd27456c64d67b22e8a58af93d93d' -> 'b290c1bf893444078ed721f25995d124' move completed: OK
I20240502 01:59:14.994084 29547 rebalancer_tool.cc:1136] tablet 5515469fdf494900ae20f20fdfca587c: '998b57e2badc4d779c586c6579149114' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:15.000373 29547 rebalancer_tool.cc:1136] tablet e1cd4681de8645ff92b479ccdd3eac37: '842c1b011b5d4c77bfd9b9591111502e' -> 'f3d18a019013487ab801599cfb80e4b4' move completed: OK
I20240502 01:59:16.070180 29547 rebalancer_tool.cc:1136] tablet 626774246b9445bc838427358e08f2cd: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:16.316694 29547 rebalancer_tool.cc:1136] tablet 78609c45dc0c4e1b838bb737ebf7e034: '68e5b1eeb04448fea818a14991eab87c' -> '998b57e2badc4d779c586c6579149114' move completed: OK
I20240502 01:59:16.324326 29547 rebalancer_tool.cc:1136] tablet 089d9dc33a594712ac04ad28a0cd37b1: '7c693d0933c64d9babc07f404df45cd1' -> '842c1b011b5d4c77bfd9b9591111502e' move completed: OK
I20240502 01:59:16.330008 29547 rebalancer_tool.cc:1136] tablet 499d2ca7bffc42748591db5280f829f2: '468dd27456c64d67b22e8a58af93d93d' -> 'b290c1bf893444078ed721f25995d124' move completed: OK
I20240502 01:59:16.353663 29547 rebalancer_tool.cc:1074] tablet dc99f8bfb06c4bcda6111b218125826c: '998b57e2badc4d779c586c6579149114' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:16.394204 29547 rebalancer_tool.cc:1074] tablet 398742b2cb3e41f1bc16fa37a5168809: 'f3d18a019013487ab801599cfb80e4b4' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:59:16.427760 29547 rebalancer_tool.cc:1074] tablet b265eec55ee1406b9ff99c159304fa65: '842c1b011b5d4c77bfd9b9591111502e' -> '468dd27456c64d67b22e8a58af93d93d' move scheduled
I20240502 01:59:16.460045 29547 rebalancer_tool.cc:1074] tablet 02e988cd07184e97921e7376da20f251: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '998b57e2badc4d779c586c6579149114' move scheduled
I20240502 01:59:16.496464 29547 rebalancer_tool.cc:1074] tablet 565aab51035c4f328e49eca53eddf621: 'b290c1bf893444078ed721f25995d124' -> '7c693d0933c64d9babc07f404df45cd1' move scheduled
I20240502 01:59:16.522012 29547 rebalancer_tool.cc:1074] tablet e66b19b5551b4011bf1bbb6234a2aba3: '68e5b1eeb04448fea818a14991eab87c' -> 'f3d18a019013487ab801599cfb80e4b4' move scheduled
I20240502 01:59:16.547389 29547 rebalancer_tool.cc:1074] tablet 11a922c5a8994e92a4112d89ce8a9f3a: '468dd27456c64d67b22e8a58af93d93d' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
I20240502 01:59:16.578809 29547 rebalancer_tool.cc:1074] tablet e96f6477ca7a497c82c01ef0b2d2747f: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '998b57e2badc4d779c586c6579149114' move scheduled
I20240502 01:59:16.606874 29547 rebalancer_tool.cc:1074] tablet 4caef235274947e48765e1c11669112a: '7c693d0933c64d9babc07f404df45cd1' -> 'b290c1bf893444078ed721f25995d124' move scheduled
I20240502 01:59:16.634821 29547 rebalancer_tool.cc:1074] tablet 12420fbcddc5446b857d2e6a01f40aaa: 'f3d18a019013487ab801599cfb80e4b4' -> '68e5b1eeb04448fea818a14991eab87c' move scheduled
I20240502 01:59:16.658938 29547 rebalancer_tool.cc:1074] tablet 68f002fcea6f4378954808dbfb5f10e6: '468dd27456c64d67b22e8a58af93d93d' -> '842c1b011b5d4c77bfd9b9591111502e' move scheduled
W20240502 01:59:17.125320 7903 connection.cc:535] client connection to 127.19.43.200:36259 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:17.130061 7956 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 01:59:17.125851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.43.200:36259 (local address 127.0.0.1:58576)
0502 01:59:17.126050 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0502 01:59:17.126066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0502 01:59:17.126151 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 01:59:17.129299 (+ 3148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":97}
I20240502 01:59:17.748867 29547 rebalancer_tool.cc:1123] tablet e66b19b5551b4011bf1bbb6234a2aba3: '68e5b1eeb04448fea818a14991eab87c' -> 'f3d18a019013487ab801599cfb80e4b4' move is abandoned: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111)
I20240502 01:59:19.380421 29547 rebalancer_tool.cc:1136] tablet e1950ace91824415b9bf3d1657bad72f: '842c1b011b5d4c77bfd9b9591111502e' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:19.386546 29547 rebalancer_tool.cc:1136] tablet 9ca8563ff68c4fa590c17ef58e67afa0: '468dd27456c64d67b22e8a58af93d93d' -> '68e5b1eeb04448fea818a14991eab87c' move completed: OK
I20240502 01:59:19.394140 29547 rebalancer_tool.cc:1136] tablet d216b8ccd8ea43df85c1013cdaf36564: '998b57e2badc4d779c586c6579149114' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:19.399734 29547 rebalancer_tool.cc:1136] tablet b546a99821624f0588a4c0f59a5ec5a3: '68e5b1eeb04448fea818a14991eab87c' -> '7c693d0933c64d9babc07f404df45cd1' move completed: OK
I20240502 01:59:19.405485 29547 rebalancer_tool.cc:1136] tablet 8209cefcc49042b18fdd97d845d167e2: '7c693d0933c64d9babc07f404df45cd1' -> '68e5b1eeb04448fea818a14991eab87c' move completed: OK
I20240502 01:59:19.410794 29547 rebalancer_tool.cc:1136] tablet 273b6d65e3df42ffb072e2129e832a92: '468dd27456c64d67b22e8a58af93d93d' -> 'b290c1bf893444078ed721f25995d124' move completed: OK
I20240502 01:59:19.415933 29547 rebalancer_tool.cc:1136] tablet 8afd4c5362464c4cba43d5ef28b50c29: 'f3d18a019013487ab801599cfb80e4b4' -> '0d83fdd1e6e04cf5a5602994d1dbc0df' move completed: OK
I20240502 01:59:19.422129 29547 rebalancer_tool.cc:1136] tablet 7f701a3437f84a84b6e3290266d24c94: 'f3d18a019013487ab801599cfb80e4b4' -> '842c1b011b5d4c77bfd9b9591111502e' move completed: OK
I20240502 01:59:19.447378 29547 rebalancer_tool.cc:1074] tablet 1b4f8eed739249cd9fc125d5ef5a6ed8: '0d83fdd1e6e04cf5a5602994d1dbc0df' -> '68e5b1eeb04448fea818a14991eab87c' move scheduled
I20240502 01:59:20.347420 29547 rebalancer_tool.cc:1074] tablet bb13643f981847c4bb480b74a4764c4c: '7c693d0933c64d9babc07f404df45cd1' -> '998b57e2badc4d779c586c6579149114' move scheduled
I20240502 01:59:20.348532 29547 rebalancer_tool.cc:1084] tablet d3c46255597d449ebb64f8e8d598ddba: 'f3d18a019013487ab801599cfb80e4b4' -> '468dd27456c64d67b22e8a58af93d93d' move ignored: Not found: d3c46255597d449ebb64f8e8d598ddba: tablet not found
I20240502 01:59:20.348600 29547 rebalancer_tool.cc:756] re-synchronizing cluster state
Illegal state: tablet server 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): unacceptable health status UNAVAILABLE
I20240502 01:59:20.638412 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 26577
W20240502 01:59:20.646476 29149 connection.cc:535] client connection to 127.19.43.193:32929 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 01:59:20.647274 29149 consensus_peers.cc:485] T 071041f8947347aa81e02721ed08731b P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.193:32929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:20.647342 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 26711
W20240502 01:59:20.650206 26997 consensus_peers.cc:485] T 936e77ced55c41d6bfe955d9a4271b80 P 68e5b1eeb04448fea818a14991eab87c -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.193:32929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 01:59:20.655882 29150 consensus_peers.cc:485] T 7d6c35112da74b34b7c71318c61fda5e P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 7c693d0933c64d9babc07f404df45cd1 (127.19.43.200:36259): Couldn't send request to peer 7c693d0933c64d9babc07f404df45cd1. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.200:36259: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 01:59:20.656414 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 26844
W20240502 01:59:20.665980 29285 consensus_peers.cc:485] T 0dc3e48cb92b4788a88c3cbf715d1596 P 468dd27456c64d67b22e8a58af93d93d -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.193:32929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:20.666085 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 26980
W20240502 01:59:20.672808 29150 connection.cc:535] client connection to 127.19.43.196:43635 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 01:59:20.673427 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 28997
W20240502 01:59:20.674973 29150 consensus_peers.cc:485] T d659e57b5ed94e30b3a821c896e974a0 P 0d83fdd1e6e04cf5a5602994d1dbc0df -> Peer 68e5b1eeb04448fea818a14991eab87c (127.19.43.196:43635): Couldn't send request to peer 68e5b1eeb04448fea818a14991eab87c. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.196:43635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:20.681394 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 29131
W20240502 01:59:20.683264 29285 consensus_peers.cc:485] T aa5b1e209f3c486e94bda3dd2999abd3 P 468dd27456c64d67b22e8a58af93d93d -> Peer 998b57e2badc4d779c586c6579149114 (127.19.43.193:32929): Couldn't send request to peer 998b57e2badc4d779c586c6579149114. Status: Network error: Client connection negotiation failed: client connection to 127.19.43.193:32929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 01:59:20.689406 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 29267
I20240502 01:59:20.695762 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 26484
2024-05-02T01:59:20Z chronyd exiting
I20240502 01:59:20.711789 19631 test_util.cc:175] -----------------------------------------------
I20240502 01:59:20.711827 19631 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714615075986586-19631-0
[ FAILED ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0, where GetParam() = 4-byte object <00-00 00-00> (51334 ms)
[----------] 1 test from DDLDuringRebalancingTest (51334 ms total)
[----------] 1 test from RebalancingDuringElectionStormTest
[ RUN ] RebalancingDuringElectionStormTest.RoundRobin/0
I20240502 01:59:20.712356 19631 test_util.cc:268] Using random seed: -208360286
I20240502 01:59:20.713030 19631 ts_itest-base.cc:112] Starting cluster with:
I20240502 01:59:20.713064 19631 ts_itest-base.cc:113] --------------
I20240502 01:59:20.713073 19631 ts_itest-base.cc:114] 8 tablet servers
I20240502 01:59:20.713081 19631 ts_itest-base.cc:115] 3 replicas per TS
I20240502 01:59:20.713090 19631 ts_itest-base.cc:116] --------------
2024-05-02T01:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T01:59:20Z Disabled control of system clock
I20240502 01:59:20.720212 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:35279
--webserver_interface=127.19.43.254
--webserver_port=0
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.43.254:35279
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 01:59:20.818997 8847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:20.819180 8847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:20.819217 8847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:20.822067 8847 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:20.822115 8847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 01:59:20.822139 8847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:20.822160 8847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 01:59:20.822175 8847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 01:59:20.825560 8847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.19.43.254:35279
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.43.254:35279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.19.43.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.8847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:20.826293 8847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:20.827273 8847 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:20.833601 8855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:20.833629 8852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:20.833721 8847 server_base.cc:1034] running on GCE node
W20240502 01:59:20.833837 8853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:20.834312 8847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:20.835126 8847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:20.836323 8847 hybrid_clock.cc:648] HybridClock initialized: now 1714615160836301 us; error 35 us; skew 500 ppm
I20240502 01:59:20.837972 8847 webserver.cc:460] Webserver started at http://127.19.43.254:43773/ using document root <none> and password file <none>
I20240502 01:59:20.838490 8847 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:20.838548 8847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:20.838704 8847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:20.840195 8847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e9af47b1014c48e7826159c5abe1cc56"
format_stamp: "Formatted at 2024-05-02 01:59:20 on dist-test-slave-mnd3"
I20240502 01:59:20.840634 8847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e9af47b1014c48e7826159c5abe1cc56"
format_stamp: "Formatted at 2024-05-02 01:59:20 on dist-test-slave-mnd3"
I20240502 01:59:20.843621 8847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:59:20.845554 8861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:20.846501 8847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:20.846596 8847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e9af47b1014c48e7826159c5abe1cc56"
format_stamp: "Formatted at 2024-05-02 01:59:20 on dist-test-slave-mnd3"
I20240502 01:59:20.846662 8847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:20.873919 8847 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:20.874455 8847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:20.874573 8847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:20.880798 8847 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.254:35279
I20240502 01:59:20.880805 8915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.254:35279 every 8 connection(s)
I20240502 01:59:20.881631 8847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 01:59:20.883916 8916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:20.884107 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 8847
I20240502 01:59:20.884224 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 01:59:20.889048 8916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56: Bootstrap starting.
I20240502 01:59:20.891165 8916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:20.891863 8916 log.cc:824] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56: Log is configured to *not* fsync() on all Append() calls
I20240502 01:59:20.893194 8916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56: No bootstrap required, opened a new log
I20240502 01:59:20.895463 8916 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9af47b1014c48e7826159c5abe1cc56" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 35279 } }
I20240502 01:59:20.895615 8916 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:20.895663 8916 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9af47b1014c48e7826159c5abe1cc56, State: Initialized, Role: FOLLOWER
I20240502 01:59:20.896004 8916 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9af47b1014c48e7826159c5abe1cc56" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 35279 } }
I20240502 01:59:20.896109 8916 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 01:59:20.896162 8916 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 01:59:20.896260 8916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:20.896867 8916 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9af47b1014c48e7826159c5abe1cc56" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 35279 } }
I20240502 01:59:20.897143 8916 leader_election.cc:302] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9af47b1014c48e7826159c5abe1cc56; no voters:
I20240502 01:59:20.897372 8916 leader_election.cc:288] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 01:59:20.897464 8921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:20.897656 8921 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [term 1 LEADER]: Becoming Leader. State: Replica: e9af47b1014c48e7826159c5abe1cc56, State: Running, Role: LEADER
I20240502 01:59:20.898245 8916 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 01:59:20.898262 8921 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9af47b1014c48e7826159c5abe1cc56" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 35279 } }
I20240502 01:59:20.899442 8922 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9af47b1014c48e7826159c5abe1cc56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9af47b1014c48e7826159c5abe1cc56" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 35279 } } }
I20240502 01:59:20.899545 8922 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [sys.catalog]: This master's current role is: LEADER
I20240502 01:59:20.899888 8932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 01:59:20.899853 8923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9af47b1014c48e7826159c5abe1cc56. Latest consensus state: current_term: 1 leader_uuid: "e9af47b1014c48e7826159c5abe1cc56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9af47b1014c48e7826159c5abe1cc56" member_type: VOTER last_known_addr { host: "127.19.43.254" port: 35279 } } }
I20240502 01:59:20.899981 8923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56 [sys.catalog]: This master's current role is: LEADER
I20240502 01:59:20.903199 8932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 01:59:20.907398 8932 catalog_manager.cc:1347] Generated new cluster ID: 51e80acef212477d8cde3da7c1dbe5b1
I20240502 01:59:20.907459 8932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 01:59:20.912547 8932 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 01:59:20.913616 8932 catalog_manager.cc:1504] Loading token signing keys...
I20240502 01:59:20.922308 8932 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e9af47b1014c48e7826159c5abe1cc56: Generated new TSK 0
I20240502 01:59:20.922955 8932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 01:59:20.932313 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.193:0
--local_ip_for_outbound_sockets=127.19.43.193
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:59:21.031443 8941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:21.031616 8941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:21.031664 8941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:21.034554 8941 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:21.034605 8941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:21.034683 8941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.193
I20240502 01:59:21.038079 8941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.19.43.193
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.8941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.193
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:21.038794 8941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:21.039742 8941 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:21.046847 8949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.046828 8946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.046836 8947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.046978 8941 server_base.cc:1034] running on GCE node
I20240502 01:59:21.047386 8941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:21.047863 8941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:21.049015 8941 hybrid_clock.cc:648] HybridClock initialized: now 1714615161049003 us; error 30 us; skew 500 ppm
I20240502 01:59:21.050729 8941 webserver.cc:460] Webserver started at http://127.19.43.193:43553/ using document root <none> and password file <none>
I20240502 01:59:21.051230 8941 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:21.051286 8941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:21.051453 8941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:21.052955 8941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "d6bd594b464a44f89a636ca1b758aefb"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.053386 8941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "d6bd594b464a44f89a636ca1b758aefb"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.056356 8941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:59:21.058367 8955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.059288 8941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:21.059386 8941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "d6bd594b464a44f89a636ca1b758aefb"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.059468 8941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:21.070425 8941 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:21.070896 8941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:21.071031 8941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:21.071470 8941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:21.072243 8941 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:21.072289 8941 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.072336 8941 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:21.072364 8941 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.080080 8941 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.193:36545
I20240502 01:59:21.080103 9068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.193:36545 every 8 connection(s)
I20240502 01:59:21.080870 8941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 01:59:21.087071 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 8941
I20240502 01:59:21.087180 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 01:59:21.088987 9069 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:21.089183 9069 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:21.089190 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.194:0
--local_ip_for_outbound_sockets=127.19.43.194
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:21.089573 9069 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:21.090842 8878 ts_manager.cc:194] Registered new tserver with Master: d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:21.091918 8878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.193:50469
W20240502 01:59:21.187199 9072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:21.187381 9072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:21.187438 9072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:21.190321 9072 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:21.190375 9072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:21.190456 9072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.194
I20240502 01:59:21.193802 9072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.19.43.194
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.194
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:21.194521 9072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:21.195470 9072 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:21.202512 9079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.202499 9081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.202502 9078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.202714 9072 server_base.cc:1034] running on GCE node
I20240502 01:59:21.203034 9072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:21.203485 9072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:21.204635 9072 hybrid_clock.cc:648] HybridClock initialized: now 1714615161204625 us; error 24 us; skew 500 ppm
I20240502 01:59:21.206260 9072 webserver.cc:460] Webserver started at http://127.19.43.194:33047/ using document root <none> and password file <none>
I20240502 01:59:21.206733 9072 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:21.206789 9072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:21.206936 9072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:21.208397 9072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.208819 9072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.211746 9072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:59:21.213721 9088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.214623 9072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:21.214730 9072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.214802 9072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:21.234050 9072 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:21.234512 9072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:21.234644 9072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:21.235080 9072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:21.235832 9072 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:21.235877 9072 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.235924 9072 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:21.235952 9072 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.243479 9072 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.194:35031
I20240502 01:59:21.243499 9205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.194:35031 every 8 connection(s)
I20240502 01:59:21.244318 9072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 01:59:21.252022 9207 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:21.252202 9207 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:21.252565 9207 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:21.253383 8878 ts_manager.cc:194] Registered new tserver with Master: efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:21.253855 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9072
I20240502 01:59:21.253895 8878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.194:41085
I20240502 01:59:21.253959 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 01:59:21.255801 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.195:0
--local_ip_for_outbound_sockets=127.19.43.195
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:59:21.352705 9211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:21.352877 9211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:21.352932 9211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:21.355799 9211 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:21.355857 9211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:21.355938 9211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.195
I20240502 01:59:21.359346 9211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.19.43.195
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.195
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:21.360028 9211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:21.360961 9211 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:21.367798 9217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.367805 9216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.367980 9211 server_base.cc:1034] running on GCE node
W20240502 01:59:21.368094 9219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.368420 9211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:21.368868 9211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:21.370018 9211 hybrid_clock.cc:648] HybridClock initialized: now 1714615161370001 us; error 28 us; skew 500 ppm
I20240502 01:59:21.371609 9211 webserver.cc:460] Webserver started at http://127.19.43.195:42911/ using document root <none> and password file <none>
I20240502 01:59:21.372059 9211 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:21.372115 9211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:21.372269 9211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:21.373680 9211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.374142 9211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.377094 9211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:21.379832 9225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.380734 9211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:21.380825 9211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.380901 9211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:21.391906 9211 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:21.392339 9211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:21.392481 9211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:21.392904 9211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:21.393648 9211 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:21.393692 9211 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.393738 9211 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:21.393764 9211 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.401460 9211 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.195:46679
I20240502 01:59:21.401487 9339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.195:46679 every 8 connection(s)
I20240502 01:59:21.402256 9211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 01:59:21.409636 9340 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:21.409807 9340 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:21.410197 9340 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:21.410218 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9211
I20240502 01:59:21.410290 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 01:59:21.411252 8877 ts_manager.cc:194] Registered new tserver with Master: ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:21.411870 8877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.195:34605
I20240502 01:59:21.412164 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.196:0
--local_ip_for_outbound_sockets=127.19.43.196
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:59:21.509692 9343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:21.509865 9343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:21.509918 9343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:21.512914 9343 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:21.512965 9343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:21.513044 9343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.196
I20240502 01:59:21.516407 9343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.19.43.196
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.196
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:21.517084 9343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:21.518049 9343 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:21.524886 9351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.524987 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.525048 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.525195 9343 server_base.cc:1034] running on GCE node
I20240502 01:59:21.525493 9343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:21.525924 9343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:21.527060 9343 hybrid_clock.cc:648] HybridClock initialized: now 1714615161527054 us; error 32 us; skew 500 ppm
I20240502 01:59:21.528618 9343 webserver.cc:460] Webserver started at http://127.19.43.196:37785/ using document root <none> and password file <none>
I20240502 01:59:21.529068 9343 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:21.529122 9343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:21.529265 9343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:21.530705 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "8d72715586b447d69353ef2fdab02ae4"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.531128 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "8d72715586b447d69353ef2fdab02ae4"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.533975 9343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:59:21.535849 9358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.536703 9343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:21.536796 9343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "8d72715586b447d69353ef2fdab02ae4"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.536870 9343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:21.564473 9343 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:21.564939 9343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:21.565073 9343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:21.565522 9343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:21.566320 9343 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:21.566371 9343 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.566419 9343 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:21.566444 9343 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.573963 9343 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.196:32843
I20240502 01:59:21.574012 9472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.196:32843 every 8 connection(s)
I20240502 01:59:21.574774 9343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 01:59:21.577179 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9343
I20240502 01:59:21.577277 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 01:59:21.579221 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:0
--local_ip_for_outbound_sockets=127.19.43.197
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:21.582656 9473 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:21.582850 9473 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:21.583228 9473 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:21.584107 8877 ts_manager.cc:194] Registered new tserver with Master: 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:21.584874 8877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.196:43619
W20240502 01:59:21.677309 9476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:21.677479 9476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:21.677520 9476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:21.680364 9476 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:21.680418 9476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:21.680483 9476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:59:21.683801 9476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:21.684458 9476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:21.685395 9476 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:21.692370 9485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.692370 9482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.692590 9483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.692518 9476 server_base.cc:1034] running on GCE node
I20240502 01:59:21.693064 9476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:21.693542 9476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:21.694674 9476 hybrid_clock.cc:648] HybridClock initialized: now 1714615161694666 us; error 25 us; skew 500 ppm
I20240502 01:59:21.696234 9476 webserver.cc:460] Webserver started at http://127.19.43.197:44281/ using document root <none> and password file <none>
I20240502 01:59:21.696702 9476 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:21.696756 9476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:21.696905 9476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:21.698400 9476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "40d8765323e84d74a0aba2fa5f4d5014"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.698822 9476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "40d8765323e84d74a0aba2fa5f4d5014"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.701695 9476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 01:59:21.703617 9491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.704458 9476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:21.704550 9476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "40d8765323e84d74a0aba2fa5f4d5014"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.704622 9476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:21.728549 9476 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:21.729007 9476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:21.729135 9476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:21.729565 9476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:21.730340 9476 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:21.730384 9476 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.730430 9476 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:21.730456 9476 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.737830 9476 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:41635
I20240502 01:59:21.737854 9608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:41635 every 8 connection(s)
I20240502 01:59:21.738628 9476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:59:21.744078 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9476
I20240502 01:59:21.744187 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 01:59:21.746317 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.198:0
--local_ip_for_outbound_sockets=127.19.43.198
--webserver_interface=127.19.43.198
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:21.746405 9609 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:21.746587 9609 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:21.747022 9609 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:21.747957 8877 ts_manager.cc:194] Registered new tserver with Master: 40d8765323e84d74a0aba2fa5f4d5014 (127.19.43.197:41635)
I20240502 01:59:21.748668 8877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:44437
W20240502 01:59:21.847831 9612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:21.847997 9612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:21.848037 9612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:21.850950 9612 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:21.851003 9612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:21.851066 9612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.198
I20240502 01:59:21.854542 9612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.19.43.198
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.198
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:21.855228 9612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:21.856163 9612 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:21.863071 9618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.863075 9617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:21.863101 9620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:21.863315 9612 server_base.cc:1034] running on GCE node
I20240502 01:59:21.863708 9612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:21.864269 9612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:21.865427 9612 hybrid_clock.cc:648] HybridClock initialized: now 1714615161865411 us; error 29 us; skew 500 ppm
I20240502 01:59:21.867053 9612 webserver.cc:460] Webserver started at http://127.19.43.198:42979/ using document root <none> and password file <none>
I20240502 01:59:21.867542 9612 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:21.867594 9612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:21.867859 9612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:21.869336 9612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "8fb7581a14e3402a8e7d44eef9615733"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.869758 9612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "8fb7581a14e3402a8e7d44eef9615733"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.872728 9612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240502 01:59:21.874727 9626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.875576 9612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:21.875675 9612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "8fb7581a14e3402a8e7d44eef9615733"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:21.875762 9612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:21.898113 9612 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:21.898578 9612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:21.898707 9612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:21.899128 9612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:21.899953 9612 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:21.899998 9612 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.900032 9612 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:21.900053 9612 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:21.907768 9612 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.198:46009
I20240502 01:59:21.907814 9742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.198:46009 every 8 connection(s)
I20240502 01:59:21.908617 9612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 01:59:21.911721 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9612
I20240502 01:59:21.911823 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 01:59:21.913870 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.199:0
--local_ip_for_outbound_sockets=127.19.43.199
--webserver_interface=127.19.43.199
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:21.916817 9743 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:21.916997 9743 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:21.917366 9743 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:21.918251 8877 ts_manager.cc:194] Registered new tserver with Master: 8fb7581a14e3402a8e7d44eef9615733 (127.19.43.198:46009)
I20240502 01:59:21.918974 8877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.198:34159
W20240502 01:59:22.014961 9746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:22.015129 9746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:22.015170 9746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:22.018011 9746 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:22.018088 9746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:22.018157 9746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.199
I20240502 01:59:22.021523 9746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.19.43.199
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.199
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:22.022224 9746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:22.023169 9746 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:22.030066 9752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:22.030104 9754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:22.030129 9751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:22.030234 9746 server_base.cc:1034] running on GCE node
I20240502 01:59:22.030639 9746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:22.031080 9746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:22.032215 9746 hybrid_clock.cc:648] HybridClock initialized: now 1714615162032203 us; error 30 us; skew 500 ppm
I20240502 01:59:22.033810 9746 webserver.cc:460] Webserver started at http://127.19.43.199:46489/ using document root <none> and password file <none>
I20240502 01:59:22.034379 9746 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:22.034440 9746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:22.034596 9746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:22.036072 9746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "dfe1786af0dc4cc4b3149d86b0623dd9"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:22.036505 9746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "dfe1786af0dc4cc4b3149d86b0623dd9"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:22.039440 9746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 01:59:22.041347 9761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:22.042212 9746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:59:22.042312 9746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "dfe1786af0dc4cc4b3149d86b0623dd9"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:22.042382 9746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:22.054663 9746 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:22.055116 9746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:22.055248 9746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:22.055680 9746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:22.056429 9746 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:22.056473 9746 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:22.056520 9746 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:22.056547 9746 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:22.064213 9746 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.199:46643
I20240502 01:59:22.064235 9879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.199:46643 every 8 connection(s)
I20240502 01:59:22.064965 9746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 01:59:22.069065 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9746
I20240502 01:59:22.069144 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 01:59:22.071173 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.200:0
--local_ip_for_outbound_sockets=127.19.43.200
--webserver_interface=127.19.43.200
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:22.072713 9880 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:22.072898 9880 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:22.073259 9880 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:22.074095 8877 ts_manager.cc:194] Registered new tserver with Master: dfe1786af0dc4cc4b3149d86b0623dd9 (127.19.43.199:46643)
I20240502 01:59:22.074609 8877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.199:47341
I20240502 01:59:22.093804 9069 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
W20240502 01:59:22.169929 9883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:22.170120 9883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:22.170176 9883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:22.173066 9883 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:22.173120 9883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:22.173208 9883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.200
I20240502 01:59:22.176558 9883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.200:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.19.43.200
--webserver_port=0
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.9883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.200
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:22.177265 9883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:22.178233 9883 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:22.185117 9889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:22.185129 9888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:22.185281 9883 server_base.cc:1034] running on GCE node
W20240502 01:59:22.185169 9891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:22.185704 9883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:22.186183 9883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:22.187335 9883 hybrid_clock.cc:648] HybridClock initialized: now 1714615162187325 us; error 22 us; skew 500 ppm
I20240502 01:59:22.188880 9883 webserver.cc:460] Webserver started at http://127.19.43.200:35325/ using document root <none> and password file <none>
I20240502 01:59:22.189342 9883 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:22.189391 9883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:22.189551 9883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 01:59:22.190989 9883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "1110b7aef73d4823bb3fa069b92a1279"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:22.191396 9883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "1110b7aef73d4823bb3fa069b92a1279"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:22.194198 9883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 01:59:22.196025 9897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:22.196863 9883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:22.196950 9883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "1110b7aef73d4823bb3fa069b92a1279"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:22.197026 9883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:22.209233 9883 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:22.209663 9883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:22.209787 9883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:22.210234 9883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:22.210968 9883 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:22.211010 9883 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:22.211058 9883 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:22.211086 9883 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:22.218693 9883 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.200:46523
I20240502 01:59:22.218714 10012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.200:46523 every 8 connection(s)
I20240502 01:59:22.219453 9883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 01:59:22.225883 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 9883
I20240502 01:59:22.225996 19631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 01:59:22.226720 10014 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:22.226928 10014 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:22.227339 10014 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:22.228166 8877 ts_manager.cc:194] Registered new tserver with Master: 1110b7aef73d4823bb3fa069b92a1279 (127.19.43.200:46523)
I20240502 01:59:22.228695 8877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.200:44347
I20240502 01:59:22.239805 19631 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 01:59:22.248453 19631 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 01:59:22.248514 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 9476
I20240502 01:59:22.255317 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 9612
I20240502 01:59:22.255909 9207 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:22.260556 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 9746
I20240502 01:59:22.265357 19631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu with pid 9883
I20240502 01:59:22.413861 9340 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:22.586548 9473 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:26.021100 8877 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:26.040189 9003 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2efeea25947f4bab37ef8a67c6d01 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.040045 9002 tablet_service.cc:1467] Processing CreateTablet for tablet 44fa89e0dff6462d8782bff0e9f06165 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.040045 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 9928d7f049d648ea84508474dd42cdc2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.041060 9002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fa89e0dff6462d8782bff0e9f06165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.041352 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 9928d7f049d648ea84508474dd42cdc2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.041275 9406 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2efeea25947f4bab37ef8a67c6d01 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.042209 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9928d7f049d648ea84508474dd42cdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.042321 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 44fa89e0dff6462d8782bff0e9f06165 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.042194 9137 tablet_service.cc:1467] Processing CreateTablet for tablet 9928d7f049d648ea84508474dd42cdc2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.042973 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fa89e0dff6462d8782bff0e9f06165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.043104 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2efeea25947f4bab37ef8a67c6d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.044622 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9928d7f049d648ea84508474dd42cdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.046610 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9928d7f049d648ea84508474dd42cdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.048246 10035 tablet_bootstrap.cc:492] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.048491 10034 tablet_bootstrap.cc:492] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.048602 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2efeea25947f4bab37ef8a67c6d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.050361 10034 tablet_bootstrap.cc:654] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.050361 10035 tablet_bootstrap.cc:654] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.050701 10036 tablet_bootstrap.cc:492] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.051098 10035 log.cc:824] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: Log is configured to *not* fsync() on all Append() calls
I20240502 01:59:26.051126 10034 log.cc:824] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb: Log is configured to *not* fsync() on all Append() calls
I20240502 01:59:26.052640 10036 tablet_bootstrap.cc:654] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.052673 10035 tablet_bootstrap.cc:492] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.052673 10034 tablet_bootstrap.cc:492] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.052821 10035 ts_tablet_manager.cc:1367] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 01:59:26.052821 10034 ts_tablet_manager.cc:1367] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:59:26.053375 10036 log.cc:824] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log is configured to *not* fsync() on all Append() calls
I20240502 01:59:26.054399 9274 tablet_service.cc:1467] Processing CreateTablet for tablet f2f2efeea25947f4bab37ef8a67c6d01 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.055009 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f2efeea25947f4bab37ef8a67c6d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.055876 10034 raft_consensus.cc:348] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.056061 10034 raft_consensus.cc:374] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.056105 10034 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.056262 10035 raft_consensus.cc:348] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.056432 10035 raft_consensus.cc:374] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.056471 10035 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.056531 10034 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.056835 10035 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.057550 10035 ts_tablet_manager.cc:1398] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 01:59:26.057858 10035 tablet_bootstrap.cc:492] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.058189 9273 tablet_service.cc:1467] Processing CreateTablet for tablet 44fa89e0dff6462d8782bff0e9f06165 (DEFAULT_TABLE table=rebalance_test_table_0 [id=c4b9b243ebe045888a13e9e647d2784e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.058429 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44fa89e0dff6462d8782bff0e9f06165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.058689 10035 tablet_bootstrap.cc:654] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.059311 10034 ts_tablet_manager.cc:1398] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 01:59:26.059515 10034 tablet_bootstrap.cc:492] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.060348 10034 tablet_bootstrap.cc:654] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.061331 10035 tablet_bootstrap.cc:492] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.061400 10035 ts_tablet_manager.cc:1367] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:26.061525 10034 tablet_bootstrap.cc:492] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.061584 10034 ts_tablet_manager.cc:1367] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:26.061888 10035 raft_consensus.cc:348] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.061970 10035 raft_consensus.cc:374] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.062000 10035 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.062065 10034 raft_consensus.cc:348] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.062163 10034 raft_consensus.cc:374] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.062148 10035 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.062193 10034 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.062332 10034 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.062414 10035 ts_tablet_manager.cc:1398] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.062582 10034 ts_tablet_manager.cc:1398] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:26.062712 10034 tablet_bootstrap.cc:492] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.063503 10034 tablet_bootstrap.cc:654] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.064249 10044 tablet_bootstrap.cc:492] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:26.064507 10034 tablet_bootstrap.cc:492] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.064572 10034 ts_tablet_manager.cc:1367] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:26.064950 10034 raft_consensus.cc:348] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.065050 10034 raft_consensus.cc:374] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.065085 10034 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.065196 10034 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.065409 10034 ts_tablet_manager.cc:1398] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:26.066011 10044 tablet_bootstrap.cc:654] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.066674 10036 tablet_bootstrap.cc:492] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.066702 10044 log.cc:824] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: Log is configured to *not* fsync() on all Append() calls
I20240502 01:59:26.066848 10036 ts_tablet_manager.cc:1367] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20240502 01:59:26.067965 10044 tablet_bootstrap.cc:492] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:26.068097 10044 ts_tablet_manager.cc:1367] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240502 01:59:26.069581 10036 raft_consensus.cc:348] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.069741 10036 raft_consensus.cc:374] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.069801 10036 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.070164 10036 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.070784 10044 raft_consensus.cc:348] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.070922 10036 ts_tablet_manager.cc:1398] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 01:59:26.070938 10044 raft_consensus.cc:374] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.070984 10044 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.071189 10036 tablet_bootstrap.cc:492] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.071342 10044 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.072008 10044 ts_tablet_manager.cc:1398] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20240502 01:59:26.072041 10036 tablet_bootstrap.cc:654] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.072294 10044 tablet_bootstrap.cc:492] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:26.073081 10044 tablet_bootstrap.cc:654] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.073164 10036 tablet_bootstrap.cc:492] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.073238 10036 ts_tablet_manager.cc:1367] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:26.073779 10036 raft_consensus.cc:348] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.073874 10036 raft_consensus.cc:374] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.073915 10036 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.074018 10036 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.074312 10036 ts_tablet_manager.cc:1398] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.075033 10044 tablet_bootstrap.cc:492] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:26.075119 10044 ts_tablet_manager.cc:1367] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:59:26.075592 10044 raft_consensus.cc:348] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.075695 10044 raft_consensus.cc:374] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.075727 10044 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.075838 10044 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.076078 10044 ts_tablet_manager.cc:1398] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 01:59:26.078269 9474 tablet.cc:2369] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:59:26.084661 9070 tablet.cc:2369] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:59:26.149745 10040 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.149909 10040 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.150892 10040 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:26.154654 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:26.154685 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:26.154914 9294 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:26.154908 9426 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:26.155313 8956 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:26.155527 10040 raft_consensus.cc:2781] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.155593 10040 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.155620 10040 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
W20240502 01:59:26.155885 9341 tablet.cc:2369] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 01:59:26.156456 10040 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.156749 10040 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:26.157048 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:26.157148 9426 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.157133 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:26.157235 9294 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.157941 9426 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:26.158054 9294 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:26.158222 8956 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:26.158396 10040 raft_consensus.cc:2781] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.158582 10040 raft_consensus.cc:686] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:26.158859 10040 consensus_queue.cc:237] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.161302 8878 catalog_manager.cc:5484] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:26.195667 10040 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.195739 10040 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.196039 10040 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:26.196347 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:26.196466 9426 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:26.196708 8956 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:26.196864 10040 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.196929 10040 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.196976 10040 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.197654 10040 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.197983 10040 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:26.198307 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:26.198397 9426 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.199200 9426 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:26.199458 8956 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:26.199616 10040 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.199693 10040 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:26.199599 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:26.199601 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:26.199817 10040 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.199864 9157 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.200909 9157 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:26.201225 8878 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:26.214841 10048 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.214928 10048 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.215747 10048 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:26.219318 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:26.219475 9023 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:26.219512 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:26.219626 9157 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:26.219831 9227 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:26.220026 10048 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.220081 10048 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.220108 10048 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.221117 10048 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.221415 10048 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:26.221704 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:26.221725 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:26.221800 9157 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.221803 9023 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.222502 9157 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:26.222560 9023 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:26.222752 9227 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.222898 10048 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.223068 10048 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:26.223366 10048 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.225756 8878 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
W20240502 01:59:26.248207 9209 tablet.cc:2369] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 01:59:26.359679 10060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:26.359870 10060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:26.362808 10060 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 01:59:26.438601 9294 raft_consensus.cc:1261] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.439962 9426 raft_consensus.cc:1261] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.440230 10053 consensus_queue.cc:1035] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.440827 10053 consensus_queue.cc:1035] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.441432 9157 raft_consensus.cc:1261] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.442451 9023 raft_consensus.cc:1261] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.443042 10048 consensus_queue.cc:1035] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.443899 9157 raft_consensus.cc:1261] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.444450 10040 consensus_queue.cc:1035] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.444478 10048 consensus_queue.cc:1035] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.445173 9426 raft_consensus.cc:1261] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.445708 10040 consensus_queue.cc:1035] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.447906 10090 mvcc.cc:205] Tried to move back new op lower bound from 7023063721724735488 to 7023063720588705792. Current Snapshot: MvccSnapshot[applied={T|T < 7023063721724735488}]
I20240502 01:59:26.452689 10093 mvcc.cc:205] Tried to move back new op lower bound from 7023063721755742208 to 7023063720853135360. Current Snapshot: MvccSnapshot[applied={T|T < 7023063721735815168}]
I20240502 01:59:26.461853 10098 mvcc.cc:205] Tried to move back new op lower bound from 7023063721753427968 to 7023063720755937280. Current Snapshot: MvccSnapshot[applied={T|T < 7023063721736663040}]
I20240502 01:59:26.472828 10099 mvcc.cc:205] Tried to move back new op lower bound from 7023063721753427968 to 7023063720755937280. Current Snapshot: MvccSnapshot[applied={T|T < 7023063721736663040}]
INSERT report
rows total: 2000
time total: 107.75 ms
time per row: 0.0538748 ms
I20240502 01:59:26.506512 8877 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:26.511250 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 9eacd063e75d4558998915f5ba599fba (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.511327 9273 tablet_service.cc:1467] Processing CreateTablet for tablet 9eacd063e75d4558998915f5ba599fba (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.511322 9406 tablet_service.cc:1467] Processing CreateTablet for tablet 9eacd063e75d4558998915f5ba599fba (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.511549 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eacd063e75d4558998915f5ba599fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.511569 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eacd063e75d4558998915f5ba599fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.511569 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eacd063e75d4558998915f5ba599fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.513242 10036 tablet_bootstrap.cc:492] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.513531 10035 tablet_bootstrap.cc:492] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.514106 10036 tablet_bootstrap.cc:654] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.514250 9273 tablet_service.cc:1467] Processing CreateTablet for tablet ecb2480d58284153a9ea5d7dc3b9960c (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.514361 10035 tablet_bootstrap.cc:654] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.514465 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb2480d58284153a9ea5d7dc3b9960c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.514940 10044 tablet_bootstrap.cc:492] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:26.515214 10036 tablet_bootstrap.cc:492] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.515288 10036 ts_tablet_manager.cc:1367] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:26.515637 10036 raft_consensus.cc:348] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.515717 10044 tablet_bootstrap.cc:654] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.515725 10036 raft_consensus.cc:374] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.515756 10036 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.515887 10036 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.516106 10036 ts_tablet_manager.cc:1398] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:26.516362 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 30fa0c3c70ef430fad68cdf679f9c2b6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.516577 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30fa0c3c70ef430fad68cdf679f9c2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.517167 10044 tablet_bootstrap.cc:492] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:26.517236 10044 ts_tablet_manager.cc:1367] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:59:26.517634 10044 raft_consensus.cc:348] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.517726 10044 raft_consensus.cc:374] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.517756 10044 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.517894 10035 tablet_bootstrap.cc:492] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.517871 10044 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.517964 10035 ts_tablet_manager.cc:1367] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:26.518164 10044 ts_tablet_manager.cc:1398] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:59:26.518319 10044 tablet_bootstrap.cc:492] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:26.518280 9138 tablet_service.cc:1467] Processing CreateTablet for tablet ecb2480d58284153a9ea5d7dc3b9960c (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.518486 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb2480d58284153a9ea5d7dc3b9960c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.518517 10035 raft_consensus.cc:348] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.518548 9406 tablet_service.cc:1467] Processing CreateTablet for tablet ecb2480d58284153a9ea5d7dc3b9960c (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.518612 10035 raft_consensus.cc:374] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.518640 10035 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.518738 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb2480d58284153a9ea5d7dc3b9960c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.518749 10035 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.518997 10035 ts_tablet_manager.cc:1398] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.519068 10044 tablet_bootstrap.cc:654] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.520195 10036 tablet_bootstrap.cc:492] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.520360 9002 tablet_service.cc:1467] Processing CreateTablet for tablet 30fa0c3c70ef430fad68cdf679f9c2b6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.520424 10035 tablet_bootstrap.cc:492] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.520610 9002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30fa0c3c70ef430fad68cdf679f9c2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.520902 9137 tablet_service.cc:1467] Processing CreateTablet for tablet 30fa0c3c70ef430fad68cdf679f9c2b6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fb43decb23f84f2f82d399fe726ce893]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.520956 10036 tablet_bootstrap.cc:654] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.521092 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30fa0c3c70ef430fad68cdf679f9c2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.522100 10035 tablet_bootstrap.cc:654] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.522711 10034 tablet_bootstrap.cc:492] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.523346 10036 tablet_bootstrap.cc:492] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.523350 10035 tablet_bootstrap.cc:492] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.523419 10036 ts_tablet_manager.cc:1367] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:59:26.523419 10035 ts_tablet_manager.cc:1367] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:59:26.523478 10034 tablet_bootstrap.cc:654] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.523787 10036 raft_consensus.cc:348] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.523800 10035 raft_consensus.cc:348] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.523881 10035 raft_consensus.cc:374] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.523881 10036 raft_consensus.cc:374] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.523912 10036 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.523912 10035 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.524012 10035 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.524012 10036 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.524235 10035 ts_tablet_manager.cc:1398] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:26.524246 10036 ts_tablet_manager.cc:1398] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:26.524778 10044 tablet_bootstrap.cc:492] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:26.524773 10034 tablet_bootstrap.cc:492] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.524848 10034 ts_tablet_manager.cc:1367] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:26.524848 10044 ts_tablet_manager.cc:1367] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:59:26.525002 10036 tablet_bootstrap.cc:492] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.525230 10044 raft_consensus.cc:348] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.525230 10034 raft_consensus.cc:348] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.525315 10044 raft_consensus.cc:374] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.525327 10034 raft_consensus.cc:374] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.525346 10044 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.525360 10034 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.525465 10044 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.525465 10034 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.525697 10044 ts_tablet_manager.cc:1398] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.525697 10034 ts_tablet_manager.cc:1398] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.525815 10036 tablet_bootstrap.cc:654] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.525846 10044 tablet_bootstrap.cc:492] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:26.526616 10044 tablet_bootstrap.cc:654] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.527889 10044 tablet_bootstrap.cc:492] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:26.527969 10044 ts_tablet_manager.cc:1367] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:26.528366 10044 raft_consensus.cc:348] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.528455 10044 raft_consensus.cc:374] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.528482 10036 tablet_bootstrap.cc:492] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.528484 10044 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.528544 10036 ts_tablet_manager.cc:1367] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:26.528612 10044 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.528877 10044 ts_tablet_manager.cc:1398] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.528906 10036 raft_consensus.cc:348] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.528995 10036 raft_consensus.cc:374] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.529031 10036 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.529136 10036 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.529351 10036 ts_tablet_manager.cc:1398] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.617583 10104 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.617683 10104 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.618184 10104 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:26.618491 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:26.618634 9158 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:26.618942 9227 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.619105 10104 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.619148 10104 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.619184 10104 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.619951 10104 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.620250 10104 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:26.620546 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:26.620661 9158 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.621459 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:26.621513 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:26.621532 9158 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:26.621606 9426 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:26.621912 9227 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.622107 10104 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.622187 10104 raft_consensus.cc:686] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:26.622315 10104 consensus_queue.cc:237] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.623670 8877 catalog_manager.cc:5484] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:26.667095 10097 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.667196 10097 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.668112 10097 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:26.671298 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:26.671411 9294 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:26.671396 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:26.671527 9023 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:26.671732 9089 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.671963 10097 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.672019 10097 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.672055 10097 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.672753 10097 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.673059 10097 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:26.673323 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:26.673339 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:26.673422 9023 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.673426 9294 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.674119 9294 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:26.674147 9023 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:26.674364 9089 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.674523 10097 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.674605 10097 raft_consensus.cc:686] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:26.674842 10097 consensus_queue.cc:237] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.676477 8877 catalog_manager.cc:5484] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:26.689438 10097 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.689528 10097 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.689899 10097 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:26.690265 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:26.690409 9294 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:26.690680 9089 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.690871 10097 raft_consensus.cc:2781] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.690918 10097 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.690944 10097 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.691635 10097 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.691924 10097 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:26.692250 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:26.692346 9294 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.693202 9294 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:26.693351 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:26.693359 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:26.693439 9089 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:26.693508 9426 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:26.693576 10097 raft_consensus.cc:2781] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.693653 10097 raft_consensus.cc:686] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:26.693782 10097 consensus_queue.cc:237] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:26.695163 8877 catalog_manager.cc:5484] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
W20240502 01:59:26.800590 10137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:26.800808 10137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:26.803773 10137 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 01:59:26.883558 9426 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.885185 9426 raft_consensus.cc:1261] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.885515 9425 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.885576 10104 consensus_queue.cc:1035] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.886467 9425 raft_consensus.cc:1261] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.886778 9158 raft_consensus.cc:1261] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.886859 9294 raft_consensus.cc:1261] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.886888 10097 consensus_queue.cc:1035] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.887292 10047 consensus_queue.cc:1035] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.887305 10104 consensus_queue.cc:1035] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.888417 9023 raft_consensus.cc:1261] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.888959 10047 consensus_queue.cc:1035] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:26.890117 9293 raft_consensus.cc:1261] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:26.895104 10047 consensus_queue.cc:1035] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 107.915 ms
time per row: 0.0539573 ms
I20240502 01:59:26.948297 8878 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:26.953076 9002 tablet_service.cc:1467] Processing CreateTablet for tablet ec2c7377d096437eb6e530e2086a67c7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.953370 9002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec2c7377d096437eb6e530e2086a67c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.953610 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 3c5477d6583246d492f1171f96cf6260 (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.953850 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c5477d6583246d492f1171f96cf6260. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.953908 9406 tablet_service.cc:1467] Processing CreateTablet for tablet ec2c7377d096437eb6e530e2086a67c7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.954150 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec2c7377d096437eb6e530e2086a67c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.954385 9404 tablet_service.cc:1467] Processing CreateTablet for tablet d82c9e2ba9224bad99c733a90acbfe3e (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.954631 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82c9e2ba9224bad99c733a90acbfe3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.955662 10034 tablet_bootstrap.cc:492] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.955935 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 3c5477d6583246d492f1171f96cf6260 (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.956005 10035 tablet_bootstrap.cc:492] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.956131 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c5477d6583246d492f1171f96cf6260. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.956519 10034 tablet_bootstrap.cc:654] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.956784 10035 tablet_bootstrap.cc:654] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.957158 9138 tablet_service.cc:1467] Processing CreateTablet for tablet d82c9e2ba9224bad99c733a90acbfe3e (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.957371 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82c9e2ba9224bad99c733a90acbfe3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.957906 10044 tablet_bootstrap.cc:492] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:26.957903 9003 tablet_service.cc:1467] Processing CreateTablet for tablet 3c5477d6583246d492f1171f96cf6260 (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.958139 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c5477d6583246d492f1171f96cf6260. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.958638 10044 tablet_bootstrap.cc:654] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.958650 10034 tablet_bootstrap.cc:492] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.958722 10034 ts_tablet_manager.cc:1367] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:26.959120 10034 raft_consensus.cc:348] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.959211 10034 raft_consensus.cc:374] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.959242 10034 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.959286 9137 tablet_service.cc:1467] Processing CreateTablet for tablet ec2c7377d096437eb6e530e2086a67c7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.959364 10034 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.959486 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec2c7377d096437eb6e530e2086a67c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.959537 10036 tablet_bootstrap.cc:492] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.959620 10034 ts_tablet_manager.cc:1398] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.960111 9001 tablet_service.cc:1467] Processing CreateTablet for tablet d82c9e2ba9224bad99c733a90acbfe3e (DEFAULT_TABLE table=rebalance_test_table_2 [id=727000f654c2414c974848495f6501e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:26.960230 10034 tablet_bootstrap.cc:492] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.960285 10036 tablet_bootstrap.cc:654] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.960332 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82c9e2ba9224bad99c733a90acbfe3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:26.961225 10034 tablet_bootstrap.cc:654] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.961922 10036 tablet_bootstrap.cc:492] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.962003 10036 ts_tablet_manager.cc:1367] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:26.962380 10036 raft_consensus.cc:348] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.962477 10036 raft_consensus.cc:374] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.962510 10036 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.962611 10036 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.962864 10036 ts_tablet_manager.cc:1398] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.963003 10036 tablet_bootstrap.cc:492] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:26.963029 10035 tablet_bootstrap.cc:492] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.963117 10035 ts_tablet_manager.cc:1367] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 01:59:26.963168 10034 tablet_bootstrap.cc:492] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.963232 10034 ts_tablet_manager.cc:1367] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 01:59:26.963508 10035 raft_consensus.cc:348] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.963598 10035 raft_consensus.cc:374] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.963629 10035 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.963625 10034 raft_consensus.cc:348] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.963706 10034 raft_consensus.cc:374] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.963732 10034 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.963788 10036 tablet_bootstrap.cc:654] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.963770 10035 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.963836 10034 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.964020 10035 ts_tablet_manager.cc:1398] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.964111 10034 ts_tablet_manager.cc:1398] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.964180 10035 tablet_bootstrap.cc:492] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.964241 10034 tablet_bootstrap.cc:492] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:26.964931 10036 tablet_bootstrap.cc:492] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:26.964975 10034 tablet_bootstrap.cc:654] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.964975 10035 tablet_bootstrap.cc:654] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.964984 10036 ts_tablet_manager.cc:1367] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:26.965260 10036 raft_consensus.cc:348] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.965335 10036 raft_consensus.cc:374] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.965353 10036 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.965436 10036 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:26.965669 10036 ts_tablet_manager.cc:1398] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.966534 10044 tablet_bootstrap.cc:492] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:26.966624 10044 ts_tablet_manager.cc:1367] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 01:59:26.967017 10035 tablet_bootstrap.cc:492] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.967043 10044 raft_consensus.cc:348] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.967171 10044 raft_consensus.cc:374] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.967209 10044 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.967314 10044 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.967439 10035 ts_tablet_manager.cc:1367] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:26.967557 10044 ts_tablet_manager.cc:1398] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.968217 10035 raft_consensus.cc:348] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.968330 10035 raft_consensus.cc:374] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.968360 10035 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.968502 10035 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.968786 10035 ts_tablet_manager.cc:1398] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:26.968941 10035 tablet_bootstrap.cc:492] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:26.969293 10034 tablet_bootstrap.cc:492] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:26.969362 10034 ts_tablet_manager.cc:1367] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:59:26.969738 10035 tablet_bootstrap.cc:654] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:26.970357 10034 raft_consensus.cc:348] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.970454 10034 raft_consensus.cc:374] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.970485 10034 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.970582 10034 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:26.970758 10035 tablet_bootstrap.cc:492] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:26.970837 10035 ts_tablet_manager.cc:1367] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:26.970863 10034 ts_tablet_manager.cc:1398] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:26.971202 10035 raft_consensus.cc:348] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.971290 10035 raft_consensus.cc:374] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:26.971321 10035 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:26.971431 10035 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.971663 10035 ts_tablet_manager.cc:1398] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:26.974249 10092 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:26.974318 10092 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.974618 10092 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:26.974884 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:26.974946 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:26.974980 9426 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:26.975050 9294 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:26.975207 8956 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:26.975342 10092 raft_consensus.cc:2781] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:26.975390 10092 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:26.975436 10092 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.976179 10092 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.976440 10092 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:26.976677 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:26.976764 9426 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.976754 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:26.976837 9294 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:26.977411 9426 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:26.977452 9294 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:26.977641 8956 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:26.977784 10092 raft_consensus.cc:2781] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:26.977921 10092 raft_consensus.cc:686] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:26.978063 10092 consensus_queue.cc:237] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:26.979311 8871 catalog_manager.cc:5484] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:27.077203 10171 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.077313 10171 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.078352 10171 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.082580 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.082711 9023 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.083015 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:27.083053 9360 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:27.083142 9158 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.083256 10171 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.083294 10171 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.083333 10171 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.084333 10171 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.084630 10171 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.085040 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:27.085103 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:27.085173 9023 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.085197 9158 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.086122 9023 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.086424 9360 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:27.086524 9158 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.086566 10171 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.086624 10171 raft_consensus.cc:686] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:27.086894 10171 consensus_queue.cc:237] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.088820 8871 catalog_manager.cc:5484] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:27.160104 10041 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.160192 10041 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.160517 10041 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:27.160859 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:27.160898 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.161000 9023 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.161000 9158 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.161248 9360 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:27.161423 10041 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.161477 10041 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.161517 10041 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.162289 10041 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.162562 10041 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:27.162879 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:27.162951 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:27.162987 9158 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.163061 9023 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.163708 9023 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.163722 9158 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.163961 9360 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:27.164110 10041 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.164196 10041 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:27.164319 10041 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.165720 8871 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:27.293802 10187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:27.294000 10187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:27.296888 10187 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 01:59:27.376319 9294 raft_consensus.cc:1261] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.376421 9426 raft_consensus.cc:1261] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.377178 9158 raft_consensus.cc:1261] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.377218 9157 raft_consensus.cc:1261] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.377838 10181 consensus_queue.cc:1035] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.378154 9022 raft_consensus.cc:1261] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.378190 10181 consensus_queue.cc:1035] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.378371 10041 consensus_queue.cc:1035] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.378553 9023 raft_consensus.cc:1261] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.378746 10041 consensus_queue.cc:1035] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.384107 10222 consensus_queue.cc:1035] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.384414 10222 consensus_queue.cc:1035] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 116.056 ms
time per row: 0.0580282 ms
I20240502 01:59:27.450460 8870 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:27.455685 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 0f04b862773348a78aa4596b0f42d573 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.455826 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 0f04b862773348a78aa4596b0f42d573 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.455917 9137 tablet_service.cc:1467] Processing CreateTablet for tablet 0f04b862773348a78aa4596b0f42d573 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.456001 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f04b862773348a78aa4596b0f42d573. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.456131 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f04b862773348a78aa4596b0f42d573. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.456166 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f04b862773348a78aa4596b0f42d573. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.456447 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 542bd878941942959df6a4c652f19c95 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.456660 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542bd878941942959df6a4c652f19c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.458297 10036 tablet_bootstrap.cc:492] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:27.458482 10044 tablet_bootstrap.cc:492] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:27.458942 10034 tablet_bootstrap.cc:492] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:27.459101 10036 tablet_bootstrap.cc:654] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.459049 9405 tablet_service.cc:1467] Processing CreateTablet for tablet ce2d727617dd49ad99f95158f6229284 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.459297 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2d727617dd49ad99f95158f6229284. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.459308 10044 tablet_bootstrap.cc:654] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.459754 10034 tablet_bootstrap.cc:654] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.459753 9404 tablet_service.cc:1467] Processing CreateTablet for tablet 542bd878941942959df6a4c652f19c95 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.459977 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542bd878941942959df6a4c652f19c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.460153 10044 tablet_bootstrap.cc:492] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:27.460223 10044 ts_tablet_manager.cc:1367] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:27.460620 10044 raft_consensus.cc:348] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.460726 10044 raft_consensus.cc:374] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.460753 10044 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.460863 10044 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.460990 10034 tablet_bootstrap.cc:492] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:27.461064 10034 ts_tablet_manager.cc:1367] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 01:59:27.461133 10044 ts_tablet_manager.cc:1398] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:27.461342 10035 tablet_bootstrap.cc:492] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:27.461481 10034 raft_consensus.cc:348] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.461603 10034 raft_consensus.cc:374] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.461637 10034 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.461750 10034 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.462010 10034 ts_tablet_manager.cc:1398] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:27.462087 9001 tablet_service.cc:1467] Processing CreateTablet for tablet ce2d727617dd49ad99f95158f6229284 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.462230 10035 tablet_bootstrap.cc:654] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.462307 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2d727617dd49ad99f95158f6229284. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.463394 10035 tablet_bootstrap.cc:492] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:27.463354 9273 tablet_service.cc:1467] Processing CreateTablet for tablet 542bd878941942959df6a4c652f19c95 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.463478 10035 ts_tablet_manager.cc:1367] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:27.463564 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542bd878941942959df6a4c652f19c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.463914 10035 raft_consensus.cc:348] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.464010 10035 raft_consensus.cc:374] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.464040 10035 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.464130 10034 tablet_bootstrap.cc:492] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:27.464143 10035 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.464401 10035 ts_tablet_manager.cc:1398] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.464557 10035 tablet_bootstrap.cc:492] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:27.464917 10034 tablet_bootstrap.cc:654] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.465878 9274 tablet_service.cc:1467] Processing CreateTablet for tablet ce2d727617dd49ad99f95158f6229284 (DEFAULT_TABLE table=rebalance_test_table_3 [id=9b64da21164a473e9e9c2f9517933a72]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.466089 10044 tablet_bootstrap.cc:492] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:27.466121 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2d727617dd49ad99f95158f6229284. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.466373 10035 tablet_bootstrap.cc:654] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.466538 10058 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.466604 10058 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.466936 10058 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.466938 10044 tablet_bootstrap.cc:654] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.467396 10034 tablet_bootstrap.cc:492] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:27.467468 10034 ts_tablet_manager.cc:1367] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:59:27.467687 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.467756 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:27.467811 9022 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:27.467861 10034 raft_consensus.cc:348] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.467941 10034 raft_consensus.cc:374] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.467989 10034 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.468084 10034 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.468356 10034 ts_tablet_manager.cc:1398] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20240502 01:59:27.468585 9227 leader_election.cc:341] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:27.468658 10035 tablet_bootstrap.cc:492] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:27.468727 10035 ts_tablet_manager.cc:1367] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:27.468750 9227 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:27.469050 10058 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.469135 10058 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.469180 10058 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.469127 10035 raft_consensus.cc:348] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.469221 10035 raft_consensus.cc:374] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.469269 10035 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.469384 10035 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.469640 10035 ts_tablet_manager.cc:1398] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.469990 10058 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.470132 10044 tablet_bootstrap.cc:492] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:27.470204 10044 ts_tablet_manager.cc:1367] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 01:59:27.470391 10058 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.470636 10044 raft_consensus.cc:348] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.470731 10044 raft_consensus.cc:374] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.470758 10044 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.470860 10044 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.470991 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:27.471102 10044 ts_tablet_manager.cc:1398] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.471117 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:27.471206 9022 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.471256 10044 tablet_bootstrap.cc:492] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
W20240502 01:59:27.471834 9227 leader_election.cc:341] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031): Illegal state: must be running to vote when last-logged opid is not known
I20240502 01:59:27.472074 10044 tablet_bootstrap.cc:654] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.472489 9022 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:27.472855 10036 tablet_bootstrap.cc:492] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:27.472947 10036 ts_tablet_manager.cc:1367] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 01:59:27.472954 9227 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:27.473387 10036 raft_consensus.cc:348] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.473503 10036 raft_consensus.cc:374] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.473537 10036 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.473651 10036 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.473683 10058 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.473821 10058 raft_consensus.cc:686] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:27.473922 10036 ts_tablet_manager.cc:1398] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.473945 10058 consensus_queue.cc:237] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.474093 10036 tablet_bootstrap.cc:492] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:27.474581 10044 tablet_bootstrap.cc:492] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:27.474649 10044 ts_tablet_manager.cc:1367] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:27.474906 10036 tablet_bootstrap.cc:654] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.475023 10044 raft_consensus.cc:348] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.475108 10044 raft_consensus.cc:374] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.475136 10044 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.475240 10044 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.475484 10044 ts_tablet_manager.cc:1398] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:27.475893 8878 catalog_manager.cc:5484] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:27.475998 10036 tablet_bootstrap.cc:492] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:27.476052 10036 ts_tablet_manager.cc:1367] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:27.476397 10036 raft_consensus.cc:348] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.476424 10239 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.476483 10036 raft_consensus.cc:374] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.476513 10036 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.476495 10239 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.476639 10036 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.476866 10036 ts_tablet_manager.cc:1398] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.477022 10239 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:27.477185 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:27.477268 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.477319 9426 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:27.477366 9022 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:27.477589 9227 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:27.477977 10239 raft_consensus.cc:2781] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.478077 10239 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.478121 10239 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.478857 10239 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.479179 10239 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:27.479480 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:27.479579 9426 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.480446 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:27.480480 9426 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:27.480553 9022 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.480737 9226 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:27.480914 10239 raft_consensus.cc:2781] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.481019 10239 raft_consensus.cc:686] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:27.481137 10239 consensus_queue.cc:237] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.481328 9022 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:27.482439 8869 catalog_manager.cc:5484] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:27.571300 10226 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.571437 10226 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.571899 10226 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:27.572350 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:27.572518 9158 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.572829 9360 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:27.572983 10226 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.573040 10226 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.573083 10226 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.573889 10226 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.574266 10226 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:27.574563 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:27.574672 9158 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.575522 9158 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.575950 9360 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:27.576094 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:27.576212 9294 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.576424 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:27.576526 9293 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.577206 9293 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.577386 10226 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.577473 10226 raft_consensus.cc:686] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:27.577608 10226 consensus_queue.cc:237] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:27.579331 8869 catalog_manager.cc:5484] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:27.648458 10097 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.648620 10097 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.649159 10097 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:27.649626 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.649663 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:27.649842 9022 raft_consensus.cc:2379] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in current term 1: Already voted for candidate ee7bf4c5836f4264b281adc40ecdac9d in this term.
I20240502 01:59:27.650228 9090 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:27.650416 10097 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.651314 10097 raft_consensus.cc:2726] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 01:59:27.694075 10245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:27.694316 10245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:27.697299 10245 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 01:59:27.771663 9158 raft_consensus.cc:1261] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.771690 9022 raft_consensus.cc:1261] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.772073 9022 raft_consensus.cc:1261] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.772691 10239 consensus_queue.cc:1035] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.772831 9293 raft_consensus.cc:1261] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.773005 10239 consensus_queue.cc:1035] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.773479 10226 consensus_queue.cc:1035] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.774595 10058 consensus_queue.cc:1035] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.774915 9157 raft_consensus.cc:1261] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.776146 10226 consensus_queue.cc:1035] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:27.786738 9426 raft_consensus.cc:1261] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:27.789595 10239 consensus_queue.cc:1035] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 93.1221 ms
time per row: 0.046561 ms
I20240502 01:59:27.830195 8870 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:27.835507 9001 tablet_service.cc:1467] Processing CreateTablet for tablet d1e3336bcbb54cb0a15f00209445713d (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.835742 9138 tablet_service.cc:1467] Processing CreateTablet for tablet d1e3336bcbb54cb0a15f00209445713d (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.835855 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e3336bcbb54cb0a15f00209445713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.836043 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e3336bcbb54cb0a15f00209445713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.836300 9137 tablet_service.cc:1467] Processing CreateTablet for tablet c22a74b9931b46acb4077befa2771906 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.836515 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c22a74b9931b46acb4077befa2771906. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.837590 10034 tablet_bootstrap.cc:492] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:27.837846 9274 tablet_service.cc:1467] Processing CreateTablet for tablet d1e3336bcbb54cb0a15f00209445713d (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.838135 10036 tablet_bootstrap.cc:492] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:27.838142 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e3336bcbb54cb0a15f00209445713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.838486 10034 tablet_bootstrap.cc:654] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.838948 10036 tablet_bootstrap.cc:654] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.839445 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 40f5ad44d23f4c6e802467fdc01cb8be (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.839680 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40f5ad44d23f4c6e802467fdc01cb8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.839848 10034 tablet_bootstrap.cc:492] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:27.839931 10034 ts_tablet_manager.cc:1367] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:27.839905 9003 tablet_service.cc:1467] Processing CreateTablet for tablet c22a74b9931b46acb4077befa2771906 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.840099 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c22a74b9931b46acb4077befa2771906. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.840207 10044 tablet_bootstrap.cc:492] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:27.840387 10034 raft_consensus.cc:348] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.840494 10034 raft_consensus.cc:374] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.840525 10034 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.840641 10034 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.840906 10034 ts_tablet_manager.cc:1398] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 01:59:27.840994 10044 tablet_bootstrap.cc:654] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.842166 9404 tablet_service.cc:1467] Processing CreateTablet for tablet c22a74b9931b46acb4077befa2771906 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.842180 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 40f5ad44d23f4c6e802467fdc01cb8be (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.842447 10034 tablet_bootstrap.cc:492] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:27.842496 10044 tablet_bootstrap.cc:492] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:27.842572 10044 ts_tablet_manager.cc:1367] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:27.842669 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40f5ad44d23f4c6e802467fdc01cb8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.843274 10034 tablet_bootstrap.cc:654] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.843271 10044 raft_consensus.cc:348] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.843365 10044 raft_consensus.cc:374] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.843392 10044 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.843499 10044 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.843750 10044 ts_tablet_manager.cc:1398] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.844843 10035 tablet_bootstrap.cc:492] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:27.845203 10036 tablet_bootstrap.cc:492] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:27.845293 10036 ts_tablet_manager.cc:1367] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 01:59:27.845614 10035 tablet_bootstrap.cc:654] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.845667 10036 raft_consensus.cc:348] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.845777 10036 raft_consensus.cc:374] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.845814 10036 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.845934 10036 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.846221 10036 ts_tablet_manager.cc:1398] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.846346 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c22a74b9931b46acb4077befa2771906. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.846400 10036 tablet_bootstrap.cc:492] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:27.846680 10035 tablet_bootstrap.cc:492] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:27.846765 10035 ts_tablet_manager.cc:1367] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:27.847198 10036 tablet_bootstrap.cc:654] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.847190 10035 raft_consensus.cc:348] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.847299 10035 raft_consensus.cc:374] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.847352 10035 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.847473 10035 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.847757 10035 ts_tablet_manager.cc:1398] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.848798 10036 tablet_bootstrap.cc:492] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:27.849236 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 40f5ad44d23f4c6e802467fdc01cb8be (DEFAULT_TABLE table=rebalance_test_table_4 [id=4dbf00d428ee4f2895eed45a684be7ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:27.849493 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40f5ad44d23f4c6e802467fdc01cb8be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:27.848881 10036 ts_tablet_manager.cc:1367] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:27.850188 10035 tablet_bootstrap.cc:492] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:27.850266 10036 raft_consensus.cc:348] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.850368 10036 raft_consensus.cc:374] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.850406 10036 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.850564 10036 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.850831 10036 ts_tablet_manager.cc:1398] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:27.850916 10035 tablet_bootstrap.cc:654] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.851835 10035 tablet_bootstrap.cc:492] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:27.851908 10035 ts_tablet_manager.cc:1367] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:27.852638 10035 raft_consensus.cc:348] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.852742 10035 raft_consensus.cc:374] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.852903 10035 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.853045 10035 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.853241 10044 tablet_bootstrap.cc:492] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:27.853834 10034 tablet_bootstrap.cc:492] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:27.853912 10034 ts_tablet_manager.cc:1367] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 01:59:27.854059 10044 tablet_bootstrap.cc:654] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.854336 10034 raft_consensus.cc:348] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.854422 10034 raft_consensus.cc:374] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.854442 10034 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.854542 10034 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.854686 10035 ts_tablet_manager.cc:1398] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:27.854794 10034 ts_tablet_manager.cc:1398] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:27.854959 10034 tablet_bootstrap.cc:492] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:27.855479 10044 tablet_bootstrap.cc:492] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:27.855554 10044 ts_tablet_manager.cc:1367] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:27.855758 10034 tablet_bootstrap.cc:654] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:27.856103 10044 raft_consensus.cc:348] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.856199 10044 raft_consensus.cc:374] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.856227 10044 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.856359 10044 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.856700 10044 ts_tablet_manager.cc:1398] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:27.857929 10034 tablet_bootstrap.cc:492] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:27.858009 10034 ts_tablet_manager.cc:1367] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:59:27.858688 10034 raft_consensus.cc:348] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.858781 10034 raft_consensus.cc:374] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:27.858825 10034 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:27.858939 10034 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.859194 10034 ts_tablet_manager.cc:1398] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:27.864073 10239 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.864162 10239 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.864503 10239 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:27.864837 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.864977 9022 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:27.865038 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:27.865171 9426 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:27.865229 9227 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:27.865398 10239 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.865450 10239 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.865492 10239 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.866313 10239 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.866636 10239 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:27.866916 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:27.867022 9426 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.867130 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:27.867224 9022 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.867981 9426 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:27.868084 9022 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:27.868263 9226 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:27.868414 10239 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.868489 10239 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:27.868618 10239 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:27.870252 8870 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:27.920619 10092 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.920760 10092 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.921147 10092 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.921546 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:27.921566 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:27.921685 9157 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:27.921722 9293 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:27.921979 8957 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:27.922171 10092 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.922228 10092 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.922266 10092 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.923081 10092 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.923410 10092 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.923707 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:27.923724 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:27.923813 9293 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.923825 9157 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.924551 9157 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:27.924563 9293 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:27.924820 8957 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:27.924981 10092 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.925069 10092 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:27.925210 10092 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.926739 8878 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:27.968541 10171 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:27.968703 10171 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.969169 10171 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.969583 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:27.969637 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:27.969739 9022 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.969771 9157 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:27.970104 9360 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:27.970261 10171 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:27.970326 10171 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:27.970371 10171 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.971158 10171 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.971446 10171 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:27.971808 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:27.971853 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:27.971912 9022 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.971954 9157 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:27.972625 9022 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.972692 9157 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:27.972892 9360 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:27.973040 10171 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:27.973126 10171 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:27.973265 10171 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:27.974947 8878 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
W20240502 01:59:28.109526 10298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:28.109802 10298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:28.112938 10298 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 01:59:28.193948 9157 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.194175 9022 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.194366 9158 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.194602 10171 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.195518 10092 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.196694 10171 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.200815 9293 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.202354 9023 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.202986 9426 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.203637 10239 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.205192 10181 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.205619 10239 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.223152 10091 mvcc.cc:205] Tried to move back new op lower bound from 7023063728916303872 to 7023063727822950400. Current Snapshot: MvccSnapshot[applied={T|T < 7023063728916303872}]
INSERT report
rows total: 2000
time total: 109.934 ms
time per row: 0.0549671 ms
I20240502 01:59:28.266983 8869 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:28.273056 9137 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5f4a0cb82e41878cbb63d99010c0a6 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.273384 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5f4a0cb82e41878cbb63d99010c0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.273371 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec3c859176746d697e47b54b2d5b9a2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.273603 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec3c859176746d697e47b54b2d5b9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.274005 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec3c859176746d697e47b54b2d5b9a2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.274005 9404 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5f4a0cb82e41878cbb63d99010c0a6 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.274318 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec3c859176746d697e47b54b2d5b9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.274574 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5f4a0cb82e41878cbb63d99010c0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.275735 10036 tablet_bootstrap.cc:492] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:28.276664 10035 tablet_bootstrap.cc:492] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:28.276726 10036 tablet_bootstrap.cc:654] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.277532 10035 tablet_bootstrap.cc:654] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.277813 10036 tablet_bootstrap.cc:492] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:28.277894 10036 ts_tablet_manager.cc:1367] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:28.277878 9273 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec3c859176746d697e47b54b2d5b9a2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.277899 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5f4a0cb82e41878cbb63d99010c0a6 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.278177 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec3c859176746d697e47b54b2d5b9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.278277 9272 tablet_service.cc:1467] Processing CreateTablet for tablet c02e74f8b37e4397aa5dc4a3e9acbe59 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.278479 10036 raft_consensus.cc:348] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.278582 10036 raft_consensus.cc:374] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.278609 10036 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.278726 10036 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.278998 10036 ts_tablet_manager.cc:1398] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:28.279037 9001 tablet_service.cc:1467] Processing CreateTablet for tablet c02e74f8b37e4397aa5dc4a3e9acbe59 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.279177 10036 tablet_bootstrap.cc:492] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:28.279309 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c02e74f8b37e4397aa5dc4a3e9acbe59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.280272 10036 tablet_bootstrap.cc:654] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.281308 10036 tablet_bootstrap.cc:492] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:28.281385 10036 ts_tablet_manager.cc:1367] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:28.281476 10034 tablet_bootstrap.cc:492] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:28.281818 10036 raft_consensus.cc:348] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.281940 10036 raft_consensus.cc:374] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.281972 10036 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.282145 10036 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.282397 10034 tablet_bootstrap.cc:654] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.282428 10036 ts_tablet_manager.cc:1398] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.283533 9137 tablet_service.cc:1467] Processing CreateTablet for tablet c02e74f8b37e4397aa5dc4a3e9acbe59 (DEFAULT_TABLE table=rebalance_test_table_5 [id=e02e41509f844df48308bbf61b02e028]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.283789 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c02e74f8b37e4397aa5dc4a3e9acbe59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.283916 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c02e74f8b37e4397aa5dc4a3e9acbe59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.284034 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5f4a0cb82e41878cbb63d99010c0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.285810 10036 tablet_bootstrap.cc:492] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:28.285970 10044 tablet_bootstrap.cc:492] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:28.286644 10036 tablet_bootstrap.cc:654] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.286897 10044 tablet_bootstrap.cc:654] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.287179 10034 tablet_bootstrap.cc:492] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:28.287278 10034 ts_tablet_manager.cc:1367] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 01:59:28.287683 10036 tablet_bootstrap.cc:492] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:28.287765 10036 ts_tablet_manager.cc:1367] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.287798 10034 raft_consensus.cc:348] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.287935 10034 raft_consensus.cc:374] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.287981 10034 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.288110 10034 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.288184 10036 raft_consensus.cc:348] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.288380 10034 ts_tablet_manager.cc:1398] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.288291 10036 raft_consensus.cc:374] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.288868 10036 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.289115 10044 tablet_bootstrap.cc:492] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:28.289083 10036 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.289196 10044 ts_tablet_manager.cc:1367] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:59:28.289628 10044 raft_consensus.cc:348] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.289727 10044 raft_consensus.cc:374] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.289759 10044 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.289870 10044 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.290140 10044 ts_tablet_manager.cc:1398] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:28.289932 10036 ts_tablet_manager.cc:1398] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.290457 10035 tablet_bootstrap.cc:492] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:28.290462 10044 tablet_bootstrap.cc:492] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:28.290561 10035 ts_tablet_manager.cc:1367] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 01:59:28.291038 10035 raft_consensus.cc:348] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.291146 10035 raft_consensus.cc:374] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.291177 10035 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.291242 10044 tablet_bootstrap.cc:654] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.291290 10035 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.291569 10035 ts_tablet_manager.cc:1398] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.291765 10035 tablet_bootstrap.cc:492] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:28.292737 10035 tablet_bootstrap.cc:654] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.292879 10044 tablet_bootstrap.cc:492] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:28.292949 10044 ts_tablet_manager.cc:1367] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:28.293332 10044 raft_consensus.cc:348] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.293433 10044 raft_consensus.cc:374] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.293464 10044 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.293581 10044 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.293840 10044 ts_tablet_manager.cc:1398] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:28.294014 10044 tablet_bootstrap.cc:492] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:28.294418 10035 tablet_bootstrap.cc:492] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:28.294493 10035 ts_tablet_manager.cc:1367] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:28.294883 10044 tablet_bootstrap.cc:654] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.294962 10035 raft_consensus.cc:348] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.295070 10035 raft_consensus.cc:374] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.295101 10035 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.295380 10035 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.295776 10035 ts_tablet_manager.cc:1398] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.297255 10044 tablet_bootstrap.cc:492] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:28.297351 10044 ts_tablet_manager.cc:1367] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 01:59:28.297732 10044 raft_consensus.cc:348] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.297868 10044 raft_consensus.cc:374] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.297950 10044 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.298125 10044 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.298378 10044 ts_tablet_manager.cc:1398] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:28.303062 10239 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:28.303181 10239 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.303550 10239 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:28.303969 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:28.303972 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:28.304119 9158 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:28.304152 9426 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:28.304445 9226 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:28.304634 10239 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:28.304723 10239 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:28.304767 10239 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.305625 10239 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.305941 10239 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:28.306293 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:28.306345 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:28.306432 9158 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.306476 9426 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.307370 9158 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:28.307430 9426 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:28.307690 9226 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:28.307875 10239 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:28.307963 10239 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:28.308101 10239 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.309836 8875 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:28.347697 10092 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:28.347843 10092 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.348294 10092 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:28.348737 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:28.348898 9158 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:28.348970 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:28.349136 9293 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:28.349246 8957 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:28.349472 10092 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:28.349526 10092 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:28.349560 10092 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.350467 10092 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.350826 10092 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:28.351151 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:28.351202 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:28.351269 9158 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.351310 9293 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.352025 9158 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:28.352308 8957 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:28.352463 10181 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:28.352543 10181 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:28.352682 10181 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.354303 8875 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:28.370894 9293 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:28.593320 10278 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:28.593485 10278 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.593925 10278 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:28.594385 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:28.594485 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:28.594552 9426 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:28.594616 9294 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:28.594888 9089 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:28.595095 10278 raft_consensus.cc:2781] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:28.595180 10278 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:28.595213 10278 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.596289 10278 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.596684 10278 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:28.597048 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:28.597184 9426 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.597170 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:28.597307 9294 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.598243 9426 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:28.598243 9294 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:28.598590 9089 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:28.598803 10278 raft_consensus.cc:2781] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:28.599009 10278 raft_consensus.cc:686] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:28.599175 10278 consensus_queue.cc:237] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.601040 8875 catalog_manager.cc:5484] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:28.719372 10239 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.723420 10058 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 01:59:28.757719 10349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:28.757980 10349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:28.761067 10349 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 01:59:28.845376 9157 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.845983 10181 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.849086 9294 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.851699 10092 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.852758 9294 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.854602 9424 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:28.856173 10390 mvcc.cc:205] Tried to move back new op lower bound from 7023063731585773568 to 7023063729574023168. Current Snapshot: MvccSnapshot[applied={T|T < 7023063731585773568}]
I20240502 01:59:28.856612 10278 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:28.856909 10347 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 108.951 ms
time per row: 0.0544757 ms
I20240502 01:59:28.910972 8876 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:28.916301 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 82a305710d4443bbac2a13f4ea8d5224 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.916568 9404 tablet_service.cc:1467] Processing CreateTablet for tablet 4f33c787846045d08c1ad152632509e7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.916639 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82a305710d4443bbac2a13f4ea8d5224. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.916814 9404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f33c787846045d08c1ad152632509e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.917035 9272 tablet_service.cc:1467] Processing CreateTablet for tablet 4f33c787846045d08c1ad152632509e7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.917282 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f33c787846045d08c1ad152632509e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.918820 9406 tablet_service.cc:1467] Processing CreateTablet for tablet 462a0918c439471791f181bf6022d6a4 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.918982 10412 tablet_bootstrap.cc:492] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:28.919066 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462a0918c439471791f181bf6022d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.919132 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 82a305710d4443bbac2a13f4ea8d5224 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.919353 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82a305710d4443bbac2a13f4ea8d5224. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.919770 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 4f33c787846045d08c1ad152632509e7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.919847 10412 tablet_bootstrap.cc:654] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.919870 9003 tablet_service.cc:1467] Processing CreateTablet for tablet 462a0918c439471791f181bf6022d6a4 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.919987 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f33c787846045d08c1ad152632509e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.920089 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462a0918c439471791f181bf6022d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.920240 9137 tablet_service.cc:1467] Processing CreateTablet for tablet 82a305710d4443bbac2a13f4ea8d5224 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.920487 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82a305710d4443bbac2a13f4ea8d5224. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.922369 10412 tablet_bootstrap.cc:492] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:28.922307 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 462a0918c439471791f181bf6022d6a4 (DEFAULT_TABLE table=rebalance_test_table_6 [id=54d6f2eea90e4fb6ad5623a5069968a1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:28.922443 10412 ts_tablet_manager.cc:1367] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:28.922556 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462a0918c439471791f181bf6022d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:28.922612 10415 tablet_bootstrap.cc:492] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:28.922837 10412 raft_consensus.cc:348] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:28.922945 10412 raft_consensus.cc:374] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.922971 10412 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.923076 10412 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:28.923341 10412 ts_tablet_manager.cc:1398] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:28.923418 10415 tablet_bootstrap.cc:654] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.923509 10412 tablet_bootstrap.cc:492] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:28.924249 10412 tablet_bootstrap.cc:654] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.924520 10415 tablet_bootstrap.cc:492] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:28.924603 10415 ts_tablet_manager.cc:1367] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.925077 10413 tablet_bootstrap.cc:492] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:28.925042 10415 raft_consensus.cc:348] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:28.925140 10415 raft_consensus.cc:374] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.925168 10415 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.925282 10415 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:28.925552 10415 ts_tablet_manager.cc:1398] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.925720 10415 tablet_bootstrap.cc:492] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:28.926502 10413 tablet_bootstrap.cc:654] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.926504 10415 tablet_bootstrap.cc:654] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.928040 10412 tablet_bootstrap.cc:492] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:28.928123 10412 ts_tablet_manager.cc:1367] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 01:59:28.928241 10413 tablet_bootstrap.cc:492] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:28.928339 10413 ts_tablet_manager.cc:1367] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:28.928663 10412 raft_consensus.cc:348] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.928753 10412 raft_consensus.cc:374] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.928714 10413 raft_consensus.cc:348] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.928782 10412 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.928793 10413 raft_consensus.cc:374] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.928820 10413 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.928930 10412 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.928944 10413 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.929206 10412 ts_tablet_manager.cc:1398] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:28.929380 10412 tablet_bootstrap.cc:492] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:28.930217 10413 ts_tablet_manager.cc:1398] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.930321 10412 tablet_bootstrap.cc:654] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.930733 10415 tablet_bootstrap.cc:492] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:28.930819 10415 ts_tablet_manager.cc:1367] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:59:28.931277 10412 tablet_bootstrap.cc:492] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:28.931264 10415 raft_consensus.cc:348] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:28.931354 10412 ts_tablet_manager.cc:1367] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:28.931375 10415 raft_consensus.cc:374] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.931408 10415 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.931519 10415 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:28.931764 10415 ts_tablet_manager.cc:1398] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.931746 10412 raft_consensus.cc:348] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:28.931849 10412 raft_consensus.cc:374] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.931875 10412 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.931970 10412 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:28.932201 10412 ts_tablet_manager.cc:1398] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:28.940078 10421 tablet_bootstrap.cc:492] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:28.940953 10421 tablet_bootstrap.cc:654] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.941854 10421 tablet_bootstrap.cc:492] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:28.941948 10421 ts_tablet_manager.cc:1367] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.942530 10421 raft_consensus.cc:348] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.942628 10421 raft_consensus.cc:374] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.942649 10421 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.942755 10421 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.942981 10421 ts_tablet_manager.cc:1398] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:28.943113 10421 tablet_bootstrap.cc:492] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:28.943907 10421 tablet_bootstrap.cc:654] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.944909 10421 tablet_bootstrap.cc:492] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:28.944993 10421 ts_tablet_manager.cc:1367] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.945423 10421 raft_consensus.cc:348] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:28.945533 10421 raft_consensus.cc:374] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.945566 10421 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.945678 10421 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:28.945947 10421 ts_tablet_manager.cc:1398] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:28.946159 10421 tablet_bootstrap.cc:492] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:28.946969 10421 tablet_bootstrap.cc:654] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:28.947949 10421 tablet_bootstrap.cc:492] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:28.948019 10421 ts_tablet_manager.cc:1367] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:28.948340 10421 raft_consensus.cc:348] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:28.948438 10421 raft_consensus.cc:374] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:28.948479 10421 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:28.948592 10421 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:28.948861 10421 ts_tablet_manager.cc:1398] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:28.964021 10171 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:28.964135 10171 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.964496 10171 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:28.964828 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:28.964895 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:28.964963 9022 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:28.965052 9294 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:28.965232 9360 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:28.965402 10171 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:28.965454 10171 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:28.965479 10171 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.966291 10171 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.966625 10171 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:28.966951 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:28.967063 9294 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.967077 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:28.967180 9022 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:28.967800 9294 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:28.967819 9022 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:28.968070 9360 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:28.968295 10171 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:28.968472 10171 raft_consensus.cc:686] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:28.968606 10171 consensus_queue.cc:237] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:28.970160 8875 catalog_manager.cc:5484] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:29.055979 10181 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:29.056123 10181 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.056560 10181 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:29.056967 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:29.057011 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:29.057132 9158 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:29.057132 9424 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:29.057407 8956 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:29.057577 10181 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:29.057634 10181 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:29.057659 10181 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.058573 10181 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.058904 10181 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:29.059219 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:29.059227 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:29.059322 9424 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.059327 9158 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.060230 9424 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:29.060231 9158 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:29.060503 8956 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:29.060652 10181 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:29.060740 10181 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:29.060878 10181 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.062618 8875 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:29.240371 10181 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:29.240515 10181 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.240962 10181 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:29.241410 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:29.241413 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:29.241581 9424 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:29.241593 9158 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:29.241906 8957 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:29.242115 10400 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:29.242177 10400 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:29.242228 10400 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.243084 10400 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.243422 10400 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:29.243675 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:29.243741 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:29.243782 9424 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.243863 9158 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.244629 9158 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:29.244705 9424 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:29.244876 8957 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:29.245039 10400 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:29.245113 10400 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:29.245234 10400 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.246886 8875 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
W20240502 01:59:29.389207 10429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:29.389403 10429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:29.392546 10429 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 01:59:29.477850 9294 raft_consensus.cc:1261] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.478271 9022 raft_consensus.cc:1261] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.478709 9158 raft_consensus.cc:1261] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.478718 9424 raft_consensus.cc:1261] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.478821 10171 consensus_queue.cc:1035] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.479290 10171 consensus_queue.cc:1035] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.479384 10400 consensus_queue.cc:1035] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.479650 10181 consensus_queue.cc:1035] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.480291 9157 raft_consensus.cc:1261] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.480643 10400 consensus_queue.cc:1035] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.484524 9426 raft_consensus.cc:1261] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.485399 10400 consensus_queue.cc:1035] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.492898 10463 mvcc.cc:205] Tried to move back new op lower bound from 7023063734185205760 to 7023063733229731840. Current Snapshot: MvccSnapshot[applied={T|T < 7023063734185205760 or (T in {7023063734185205760})}]
I20240502 01:59:29.502530 10462 mvcc.cc:205] Tried to move back new op lower bound from 7023063734185205760 to 7023063733229731840. Current Snapshot: MvccSnapshot[applied={T|T < 7023063734185205760}]
INSERT report
rows total: 2000
time total: 100.652 ms
time per row: 0.0503261 ms
I20240502 01:59:29.537474 8871 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:29.542943 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 133e3b04d21a4223b80dfeae975ca78f (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.543262 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133e3b04d21a4223b80dfeae975ca78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.543565 9272 tablet_service.cc:1467] Processing CreateTablet for tablet 133e3b04d21a4223b80dfeae975ca78f (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.543800 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133e3b04d21a4223b80dfeae975ca78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.545325 10489 tablet_bootstrap.cc:492] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:29.545675 9137 tablet_service.cc:1467] Processing CreateTablet for tablet cf038eed2c6645d8a7dbf126e4013b97 (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.545789 9003 tablet_service.cc:1467] Processing CreateTablet for tablet 133e3b04d21a4223b80dfeae975ca78f (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.545903 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf038eed2c6645d8a7dbf126e4013b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.546083 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133e3b04d21a4223b80dfeae975ca78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.546284 10489 tablet_bootstrap.cc:654] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.546679 10490 tablet_bootstrap.cc:492] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:29.547544 10490 tablet_bootstrap.cc:654] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.547916 10492 tablet_bootstrap.cc:492] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:29.548058 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf237c991ee4754817d5de7d34dc7ab (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.548269 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf237c991ee4754817d5de7d34dc7ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.548463 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf237c991ee4754817d5de7d34dc7ab (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.548463 9406 tablet_service.cc:1467] Processing CreateTablet for tablet cf038eed2c6645d8a7dbf126e4013b97 (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.548669 10492 tablet_bootstrap.cc:654] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.548772 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf237c991ee4754817d5de7d34dc7ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.549818 10490 tablet_bootstrap.cc:492] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:29.549899 10490 ts_tablet_manager.cc:1367] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:29.550413 10490 raft_consensus.cc:348] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.550522 10490 raft_consensus.cc:374] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.550554 10490 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.550668 10489 tablet_bootstrap.cc:492] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:29.550746 10489 ts_tablet_manager.cc:1367] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 01:59:29.550791 10490 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.551004 10495 tablet_bootstrap.cc:492] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:29.551107 10489 raft_consensus.cc:348] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.551208 10489 raft_consensus.cc:374] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.551237 10489 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.551362 10489 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.551625 10489 ts_tablet_manager.cc:1398] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:29.551800 10489 tablet_bootstrap.cc:492] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:29.551828 10495 tablet_bootstrap.cc:654] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.552114 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf237c991ee4754817d5de7d34dc7ab (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.552352 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf237c991ee4754817d5de7d34dc7ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.552712 10489 tablet_bootstrap.cc:654] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.552845 10495 tablet_bootstrap.cc:492] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:29.552920 10495 ts_tablet_manager.cc:1367] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:29.553359 10495 raft_consensus.cc:348] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.553449 10495 raft_consensus.cc:374] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.553478 10495 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.553493 10490 ts_tablet_manager.cc:1398] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 01:59:29.553593 10495 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.553851 10495 ts_tablet_manager.cc:1398] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:29.554348 10489 tablet_bootstrap.cc:492] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:29.554428 10489 ts_tablet_manager.cc:1367] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:29.554759 10489 raft_consensus.cc:348] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.554823 10489 raft_consensus.cc:374] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.554841 10489 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.554940 10489 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.555138 10490 tablet_bootstrap.cc:492] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:29.555176 10489 ts_tablet_manager.cc:1398] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:29.555336 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf038eed2c6645d8a7dbf126e4013b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.555939 10490 tablet_bootstrap.cc:654] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.557003 10490 tablet_bootstrap.cc:492] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:29.557075 10490 ts_tablet_manager.cc:1367] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 01:59:29.557211 10495 tablet_bootstrap.cc:492] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:29.557426 10490 raft_consensus.cc:348] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.557504 10490 raft_consensus.cc:374] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.557545 10490 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.557658 10490 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.557869 10490 ts_tablet_manager.cc:1398] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:29.558974 10495 tablet_bootstrap.cc:654] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.559705 9272 tablet_service.cc:1467] Processing CreateTablet for tablet cf038eed2c6645d8a7dbf126e4013b97 (DEFAULT_TABLE table=rebalance_test_table_7 [id=06707635e3134b1ba58d3e7b2f0e3005]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:29.559926 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf038eed2c6645d8a7dbf126e4013b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:29.560736 10495 tablet_bootstrap.cc:492] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:29.560832 10495 ts_tablet_manager.cc:1367] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 01:59:29.561283 10495 raft_consensus.cc:348] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.561388 10495 raft_consensus.cc:374] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.561430 10495 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.561551 10495 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.561842 10495 ts_tablet_manager.cc:1398] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:29.561771 10490 tablet_bootstrap.cc:492] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:29.562394 10492 tablet_bootstrap.cc:492] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:29.562724 10492 ts_tablet_manager.cc:1367] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 01:59:29.564416 10492 raft_consensus.cc:348] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.564858 10492 raft_consensus.cc:374] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.564905 10492 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.565054 10492 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.565204 10490 tablet_bootstrap.cc:654] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.565431 10492 ts_tablet_manager.cc:1398] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20240502 01:59:29.565582 10492 tablet_bootstrap.cc:492] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:29.566635 10492 tablet_bootstrap.cc:654] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:29.567245 10490 tablet_bootstrap.cc:492] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:29.567350 10490 ts_tablet_manager.cc:1367] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20240502 01:59:29.567791 10490 raft_consensus.cc:348] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.567878 10492 tablet_bootstrap.cc:492] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:29.567932 10490 raft_consensus.cc:374] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.567970 10490 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.568001 10492 ts_tablet_manager.cc:1367] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:29.568091 10490 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.568354 10490 ts_tablet_manager.cc:1398] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:29.568418 10492 raft_consensus.cc:348] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.568516 10492 raft_consensus.cc:374] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:29.568549 10492 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:29.568683 10492 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.568935 10492 ts_tablet_manager.cc:1398] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:29.641677 10351 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:29.641798 10351 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.642215 10351 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:29.642544 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:29.642674 9424 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:29.642637 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:29.642781 9158 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:29.642992 9226 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:29.643231 10351 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:29.643301 10351 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:29.643342 10351 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.644169 10351 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.644459 10351 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:29.644807 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:29.644804 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:29.644914 9424 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.644920 9158 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.645678 9424 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:29.645915 9158 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:29.645924 9226 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:29.646102 10351 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:29.646318 10351 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:29.646457 10351 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:29.648106 8875 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:29.704223 10351 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:29.704347 10351 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.704744 10351 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:29.705152 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:29.705228 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:29.705292 9158 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:29.705389 9022 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:29.705585 9227 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:29.705752 10351 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:29.705817 10351 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:29.705861 10351 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.706660 10351 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.706969 10351 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:29.707345 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:29.707371 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:29.707455 9158 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.707470 9022 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.708173 9158 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:29.708233 9022 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:29.708437 9227 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:29.708593 10351 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:29.708703 10351 raft_consensus.cc:686] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:29.708842 10351 consensus_queue.cc:237] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:29.710615 8875 catalog_manager.cc:5484] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:29.745793 10351 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:29.745908 10351 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.746322 10351 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:29.746637 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:29.746672 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:29.746758 9022 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:29.746806 9424 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:29.747035 9227 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:29.747220 10351 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:29.747285 10351 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:29.747313 10351 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.748071 10351 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.748371 10351 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:29.748749 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:29.748749 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:29.748873 9022 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.748873 9424 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:29.749794 9424 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:29.749794 9022 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:29.750087 9226 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:29.750257 10351 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:29.750329 10351 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:29.750458 10351 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:29.752180 8875 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
W20240502 01:59:29.881561 10507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:29.881789 10507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:29.884887 10507 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 01:59:29.968632 9158 raft_consensus.cc:1261] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.969089 9023 raft_consensus.cc:1261] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.970414 10351 consensus_queue.cc:1035] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.970727 10351 consensus_queue.cc:1035] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.970782 9424 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.970873 9158 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.971186 10351 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.975004 10502 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.975978 9023 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.976637 9426 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:29.979785 10351 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:29.980026 10351 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 111.271 ms
time per row: 0.0556353 ms
I20240502 01:59:30.035688 8873 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:30.040848 9137 tablet_service.cc:1467] Processing CreateTablet for tablet 06bfc16dee5546a0a33fe56c32a4bd02 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.040809 9272 tablet_service.cc:1467] Processing CreateTablet for tablet 06bfc16dee5546a0a33fe56c32a4bd02 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.041011 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 06bfc16dee5546a0a33fe56c32a4bd02 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.041124 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06bfc16dee5546a0a33fe56c32a4bd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.041121 9137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06bfc16dee5546a0a33fe56c32a4bd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.041313 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06bfc16dee5546a0a33fe56c32a4bd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.041893 9138 tablet_service.cc:1467] Processing CreateTablet for tablet 196e584cf0b74034a99def7a33872965 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.042136 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 196e584cf0b74034a99def7a33872965. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.043692 10490 tablet_bootstrap.cc:492] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:30.043645 9136 tablet_service.cc:1467] Processing CreateTablet for tablet 9b28feae4b68470097028024d908ed23 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.043880 9136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b28feae4b68470097028024d908ed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.044035 10489 tablet_bootstrap.cc:492] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:30.044440 10490 tablet_bootstrap.cc:654] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.044924 10489 tablet_bootstrap.cc:654] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.045342 10490 tablet_bootstrap.cc:492] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:30.045421 10490 ts_tablet_manager.cc:1367] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 01:59:30.045504 10492 tablet_bootstrap.cc:492] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:30.045755 10490 raft_consensus.cc:348] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.045859 10490 raft_consensus.cc:374] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.045892 10490 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.046011 10490 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.046334 10492 tablet_bootstrap.cc:654] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.046342 10490 ts_tablet_manager.cc:1398] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:30.047282 10492 tablet_bootstrap.cc:492] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:30.047375 10492 ts_tablet_manager.cc:1367] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:30.047816 10492 raft_consensus.cc:348] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.047915 10492 raft_consensus.cc:374] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.047945 10492 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.048034 9406 tablet_service.cc:1467] Processing CreateTablet for tablet 196e584cf0b74034a99def7a33872965 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.048054 9272 tablet_service.cc:1467] Processing CreateTablet for tablet 9b28feae4b68470097028024d908ed23 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.048054 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 9b28feae4b68470097028024d908ed23 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.048069 10492 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.048274 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b28feae4b68470097028024d908ed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.048310 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 196e584cf0b74034a99def7a33872965. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.048331 10492 ts_tablet_manager.cc:1398] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:30.048406 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b28feae4b68470097028024d908ed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.049999 10490 tablet_bootstrap.cc:492] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:30.050271 10495 tablet_bootstrap.cc:492] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:30.050505 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 196e584cf0b74034a99def7a33872965 (DEFAULT_TABLE table=rebalance_test_table_8 [id=fb8109b47c6e457eb4833e58d52b686c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.050712 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 196e584cf0b74034a99def7a33872965. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.050912 10490 tablet_bootstrap.cc:654] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.051054 10495 tablet_bootstrap.cc:654] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.052767 10495 tablet_bootstrap.cc:492] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:30.052776 10492 tablet_bootstrap.cc:492] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:30.052856 10495 ts_tablet_manager.cc:1367] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 01:59:30.053233 10495 raft_consensus.cc:348] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.053346 10495 raft_consensus.cc:374] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.053375 10495 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.053480 10495 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.053539 10489 tablet_bootstrap.cc:492] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:30.053612 10489 ts_tablet_manager.cc:1367] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 01:59:30.053629 10492 tablet_bootstrap.cc:654] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.053747 10495 ts_tablet_manager.cc:1398] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 01:59:30.053905 10495 tablet_bootstrap.cc:492] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:30.054126 10489 raft_consensus.cc:348] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.054227 10489 raft_consensus.cc:374] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.054258 10489 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.054371 10489 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.054638 10489 ts_tablet_manager.cc:1398] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:30.054654 10495 tablet_bootstrap.cc:654] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.054795 10489 tablet_bootstrap.cc:492] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:30.055639 10489 tablet_bootstrap.cc:654] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.056440 10492 tablet_bootstrap.cc:492] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:30.056535 10492 ts_tablet_manager.cc:1367] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 01:59:30.056630 10489 tablet_bootstrap.cc:492] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:30.056710 10489 ts_tablet_manager.cc:1367] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:30.056947 10495 tablet_bootstrap.cc:492] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:30.057016 10495 ts_tablet_manager.cc:1367] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 01:59:30.057099 10492 raft_consensus.cc:348] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.057123 10489 raft_consensus.cc:348] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.057199 10492 raft_consensus.cc:374] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.057209 10489 raft_consensus.cc:374] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.057232 10492 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.057236 10489 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.057334 10492 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.057343 10489 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.057404 10495 raft_consensus.cc:348] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.057478 10495 raft_consensus.cc:374] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.057505 10495 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.057595 10489 ts_tablet_manager.cc:1398] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:30.057595 10492 ts_tablet_manager.cc:1398] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:30.057610 10495 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.057731 10489 tablet_bootstrap.cc:492] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:30.057868 10495 ts_tablet_manager.cc:1398] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:30.058596 10489 tablet_bootstrap.cc:654] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.059803 10489 tablet_bootstrap.cc:492] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:30.059885 10489 ts_tablet_manager.cc:1367] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:30.060281 10489 raft_consensus.cc:348] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.060379 10489 raft_consensus.cc:374] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.060406 10489 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.060524 10489 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.060803 10489 ts_tablet_manager.cc:1398] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 01:59:30.061412 10490 tablet_bootstrap.cc:492] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:30.061484 10490 ts_tablet_manager.cc:1367] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 01:59:30.061913 10490 raft_consensus.cc:348] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.062045 10490 raft_consensus.cc:374] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.062096 10490 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.062227 10490 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.062486 10490 ts_tablet_manager.cc:1398] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 01:59:30.114830 10347 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:30.114959 10347 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.115372 10347 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:30.115710 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:30.115728 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:30.115864 9294 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:30.115870 9426 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 0.
I20240502 01:59:30.116138 9089 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:30.116303 10347 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:30.116357 10347 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:30.116398 10347 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.117293 10347 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.117578 10347 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:30.117902 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:30.117949 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:30.118047 9294 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.118068 9426 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.118724 9294 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:30.118871 9426 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 1.
I20240502 01:59:30.118984 9089 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:30.119240 10347 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:30.119406 10347 raft_consensus.cc:686] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:30.119527 10347 consensus_queue.cc:237] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.121042 8876 catalog_manager.cc:5484] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 1 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:30.243404 10400 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:30.243539 10400 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.243999 10400 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:30.244412 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:30.244426 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:30.244562 9157 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:30.244570 9426 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 0.
I20240502 01:59:30.244872 8956 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:30.245059 10400 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:30.245110 10400 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:30.245141 10400 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.246191 10400 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.246568 10400 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:30.246901 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:30.246912 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:30.247004 9157 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.247015 9426 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.247840 9157 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:30.247905 9426 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 1.
I20240502 01:59:30.248152 8957 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:30.248365 10400 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:30.248442 10400 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:30.248577 10400 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:30.250398 8876 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 0 to 1, leader changed from <none> to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 1 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:30.379798 10535 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:30.379917 10535 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.380273 10535 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:30.380651 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:30.380671 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:30.380786 9157 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:30.380808 9022 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:30.381053 9227 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:30.381220 10535 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:30.381285 10535 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:30.381330 10535 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.382215 10535 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.382517 10535 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:30.382830 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:30.382885 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:30.382941 9157 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.383000 9022 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.383689 9157 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:30.383832 9022 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:30.383985 9227 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:30.384159 10535 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:30.384245 10535 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:30.384392 10535 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:30.386138 8876 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:30.517082 10557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:30.517325 10557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:30.520411 10557 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 01:59:30.562189 10347 consensus_queue.cc:1035] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:30.566416 10553 consensus_queue.cc:1035] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:30.602324 9022 raft_consensus.cc:1261] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:30.602630 9157 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:30.603071 10400 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:30.605602 10502 consensus_queue.cc:1035] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:30.607026 9158 raft_consensus.cc:1261] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:30.607435 10502 consensus_queue.cc:1035] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:30.625989 9426 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:30.629063 10400 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 265.043 ms
time per row: 0.132522 ms
I20240502 01:59:30.839104 8875 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51362:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 01:59:30.856792 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 877896deb7564195b8683801b048c80c (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.856793 9272 tablet_service.cc:1467] Processing CreateTablet for tablet 877896deb7564195b8683801b048c80c (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.857508 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 877896deb7564195b8683801b048c80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.857508 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 877896deb7564195b8683801b048c80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.857468 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 74c11fb113b44a9eb9c2370d7c88760b (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.857888 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c11fb113b44a9eb9c2370d7c88760b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.859411 9003 tablet_service.cc:1467] Processing CreateTablet for tablet 74c11fb113b44a9eb9c2370d7c88760b (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.859935 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c11fb113b44a9eb9c2370d7c88760b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.864418 10625 tablet_bootstrap.cc:492] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:30.864229 9406 tablet_service.cc:1467] Processing CreateTablet for tablet 877896deb7564195b8683801b048c80c (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.865080 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 877896deb7564195b8683801b048c80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.866014 10626 tablet_bootstrap.cc:492] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:30.867314 10625 tablet_bootstrap.cc:654] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.868284 10626 tablet_bootstrap.cc:654] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.869701 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 7ecbaecc48414a56a09af563e081362c (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.870440 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ecbaecc48414a56a09af563e081362c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.871619 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 74c11fb113b44a9eb9c2370d7c88760b (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.872059 10626 tablet_bootstrap.cc:492] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:30.872462 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c11fb113b44a9eb9c2370d7c88760b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.872462 10626 ts_tablet_manager.cc:1367] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240502 01:59:30.873724 9136 tablet_service.cc:1467] Processing CreateTablet for tablet 7ecbaecc48414a56a09af563e081362c (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.874656 9136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ecbaecc48414a56a09af563e081362c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.874588 10626 raft_consensus.cc:348] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.875401 10626 raft_consensus.cc:374] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.875594 10626 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.876356 10626 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.877647 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 7ecbaecc48414a56a09af563e081362c (DEFAULT_TABLE table=rebalance_test_table_9 [id=fcb589db25d84a42b9f5a8311fde49ab]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 01:59:30.879024 10626 ts_tablet_manager.cc:1398] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240502 01:59:30.879630 10629 tablet_bootstrap.cc:492] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:30.881980 10629 tablet_bootstrap.cc:654] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.882571 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ecbaecc48414a56a09af563e081362c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 01:59:30.891073 10625 tablet_bootstrap.cc:492] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:30.891383 10625 ts_tablet_manager.cc:1367] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.001s
I20240502 01:59:30.893158 10625 raft_consensus.cc:348] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.893539 10625 raft_consensus.cc:374] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.893646 10625 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.894229 10625 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.897490 10626 tablet_bootstrap.cc:492] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:30.900636 10626 tablet_bootstrap.cc:654] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.901743 10625 ts_tablet_manager.cc:1398] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.010s user 0.002s sys 0.003s
I20240502 01:59:30.902485 10631 tablet_bootstrap.cc:492] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Bootstrap starting.
I20240502 01:59:30.904423 10631 tablet_bootstrap.cc:654] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.904747 10625 tablet_bootstrap.cc:492] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:30.906942 10626 tablet_bootstrap.cc:492] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:30.907081 10625 tablet_bootstrap.cc:654] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.907112 10626 ts_tablet_manager.cc:1367] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240502 01:59:30.908164 10629 tablet_bootstrap.cc:492] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:30.908083 10626 raft_consensus.cc:348] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.908280 10626 raft_consensus.cc:374] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.908322 10629 ts_tablet_manager.cc:1367] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.005s
I20240502 01:59:30.908349 10626 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.908604 10626 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.909389 10625 tablet_bootstrap.cc:492] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:30.909591 10625 ts_tablet_manager.cc:1367] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 01:59:30.909621 10629 raft_consensus.cc:348] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.909945 10629 raft_consensus.cc:374] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.910054 10629 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.910434 10629 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.910948 10625 raft_consensus.cc:348] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.911273 10625 raft_consensus.cc:374] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.911352 10625 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.911621 10626 ts_tablet_manager.cc:1398] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240502 01:59:30.911746 10625 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.912331 10626 tablet_bootstrap.cc:492] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: Bootstrap starting.
I20240502 01:59:30.914553 10629 ts_tablet_manager.cc:1398] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240502 01:59:30.915043 10626 tablet_bootstrap.cc:654] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.915731 10629 tablet_bootstrap.cc:492] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4: Bootstrap starting.
I20240502 01:59:30.918593 10631 tablet_bootstrap.cc:492] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: No bootstrap required, opened a new log
I20240502 01:59:30.918864 10631 ts_tablet_manager.cc:1367] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20240502 01:59:30.920564 10631 raft_consensus.cc:348] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.920989 10631 raft_consensus.cc:374] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.921283 10631 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.921605 10631 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.922292 10631 ts_tablet_manager.cc:1398] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 01:59:30.923626 10626 tablet_bootstrap.cc:492] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: No bootstrap required, opened a new log
I20240502 01:59:30.923846 10626 ts_tablet_manager.cc:1367] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20240502 01:59:30.925932 10626 raft_consensus.cc:348] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.926103 10626 raft_consensus.cc:374] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.926440 10626 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.926728 10626 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.927613 10629 tablet_bootstrap.cc:654] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.927697 10625 ts_tablet_manager.cc:1398] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20240502 01:59:30.929423 10626 ts_tablet_manager.cc:1398] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240502 01:59:30.929910 10625 tablet_bootstrap.cc:492] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d: Bootstrap starting.
I20240502 01:59:30.932848 10629 tablet_bootstrap.cc:492] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4: No bootstrap required, opened a new log
I20240502 01:59:30.933132 10629 ts_tablet_manager.cc:1367] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20240502 01:59:30.933964 10625 tablet_bootstrap.cc:654] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d: Neither blocks nor log segments found. Creating new log.
I20240502 01:59:30.934021 10629 raft_consensus.cc:348] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.934461 10629 raft_consensus.cc:374] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.934572 10629 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.935153 10629 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:30.936328 10629 ts_tablet_manager.cc:1398] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 01:59:30.938139 10625 tablet_bootstrap.cc:492] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d: No bootstrap required, opened a new log
I20240502 01:59:30.938423 10625 ts_tablet_manager.cc:1367] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20240502 01:59:30.939776 10625 raft_consensus.cc:348] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.939922 10625 raft_consensus.cc:374] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 01:59:30.939966 10625 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Initialized, Role: FOLLOWER
I20240502 01:59:30.940094 10625 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.940372 10625 ts_tablet_manager.cc:1398] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 01:59:30.963438 10535 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:30.963560 10535 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.963910 10535 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:30.964285 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" is_pre_election: true
I20240502 01:59:30.964345 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:30.964426 9157 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:30.964512 9022 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:30.964742 9227 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:30.965106 10622 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:30.965175 10622 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:30.965227 10622 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.966127 10622 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.966447 10622 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:30.966830 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:30.966943 9157 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.966970 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:30.967089 9022 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:30.967741 9157 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:30.967803 9022 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:30.968096 9227 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:30.968261 10622 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:30.968358 10622 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:30.968484 10622 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:30.970289 8878 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:31.004276 10585 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:31.004417 10585 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:31.004858 10585 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:31.005280 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:31.005286 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" is_pre_election: true
I20240502 01:59:31.005419 9294 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:31.005422 9022 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 0.
I20240502 01:59:31.005695 9360 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:31.005858 10585 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:31.005923 10585 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:31.005954 10585 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:31.006911 10585 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:31.007298 10585 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:31.007601 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:31.007639 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:31.007716 9294 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:31.007750 9022 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:31.008482 9294 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:31.008615 9022 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 1.
I20240502 01:59:31.008775 9359 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:31.008926 10585 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:31.009014 10585 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:31.009151 10585 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:31.011059 8878 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 1 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:31.112401 10622 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 01:59:31.112534 10622 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:31.112905 10622 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:31.113346 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb" is_pre_election: true
I20240502 01:59:31.113375 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4" is_pre_election: true
I20240502 01:59:31.113495 9022 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:31.113508 9426 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 0.
I20240502 01:59:31.113778 9227 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:31.113953 10622 raft_consensus.cc:2781] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 01:59:31.114012 10622 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 01:59:31.114080 10622 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:31.114887 10622 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:31.115204 10622 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:31.115530 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:31.115530 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:31.115645 9426 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:31.115645 9022 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 0 FOLLOWER]: Advancing to term 1
I20240502 01:59:31.116554 9426 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:31.116554 9022 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 1.
I20240502 01:59:31.116832 9226 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:31.116993 10622 raft_consensus.cc:2781] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 01:59:31.117079 10622 raft_consensus.cc:686] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:31.117213 10622 consensus_queue.cc:237] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:31.118930 8878 catalog_manager.cc:5484] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 1 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
W20240502 01:59:31.255283 10641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:31.255517 10641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:31.258615 10641 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 01:59:31.356747 9022 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:31.356818 9157 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:31.357218 10535 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:31.357384 10622 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:31.360649 9426 raft_consensus.cc:1261] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:31.361090 10535 consensus_queue.cc:1035] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:31.364651 9023 raft_consensus.cc:1261] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:31.366019 10535 consensus_queue.cc:1035] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:31.368492 9021 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:31.368829 9294 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 01:59:31.369231 10585 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:31.369973 10585 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 01:59:31.386309 10670 mvcc.cc:205] Tried to move back new op lower bound from 7023063741873061888 to 7023063740288086016. Current Snapshot: MvccSnapshot[applied={T|T < 7023063741873061888}]
INSERT report
rows total: 2000
time total: 124.302 ms
time per row: 0.0621509 ms
I20240502 01:59:31.433501 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.197:41635
--local_ip_for_outbound_sockets=127.19.43.197
--tserver_master_addrs=127.19.43.254:35279
--webserver_port=44281
--webserver_interface=127.19.43.197
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 01:59:31.546170 10700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:31.546394 10700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:31.546437 10700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:31.549615 10700 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:31.549679 10700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:31.549759 10700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.197
I20240502 01:59:31.553442 10700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.197:41635
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.19.43.197
--webserver_port=44281
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.10700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.197
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:31.554399 10700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:31.555562 10700 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:31.563483 10706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:31.563508 10709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:31.563601 10707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:31.563961 10700 server_base.cc:1034] running on GCE node
I20240502 01:59:31.564399 10700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:31.565002 10700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:31.566519 10700 hybrid_clock.cc:648] HybridClock initialized: now 1714615171566438 us; error 66 us; skew 500 ppm
I20240502 01:59:31.570448 10700 webserver.cc:460] Webserver started at http://127.19.43.197:44281/ using document root <none> and password file <none>
I20240502 01:59:31.570966 10700 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:31.571033 10700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:31.574188 10700 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:59:31.576156 10715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.577118 10700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:31.577219 10700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "40d8765323e84d74a0aba2fa5f4d5014"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:31.577708 10700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:31.594071 10700 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:31.594566 10700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:31.594704 10700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:31.595204 10700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:31.596062 10700 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:31.596109 10700 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.596159 10700 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:31.596184 10700 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.604032 10700 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.197:41635
I20240502 01:59:31.604050 10831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.197:41635 every 8 connection(s)
I20240502 01:59:31.604985 10700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 01:59:31.609787 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 10700
I20240502 01:59:31.610183 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.198:46009
--local_ip_for_outbound_sockets=127.19.43.198
--tserver_master_addrs=127.19.43.254:35279
--webserver_port=42979
--webserver_interface=127.19.43.198
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:31.613811 10832 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:31.614064 10832 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:31.614591 10832 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:31.615696 8871 ts_manager.cc:194] Re-registered known tserver with Master: 40d8765323e84d74a0aba2fa5f4d5014 (127.19.43.197:41635)
I20240502 01:59:31.616240 8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.197:51209
W20240502 01:59:31.715790 10835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:31.715986 10835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:31.716034 10835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:31.719040 10835 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:31.719096 10835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:31.719177 10835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.198
I20240502 01:59:31.722645 10835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.198:46009
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.19.43.198
--webserver_port=42979
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.10835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.198
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:31.723408 10835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:31.724428 10835 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:31.731869 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:31.731948 10835 server_base.cc:1034] running on GCE node
W20240502 01:59:31.731855 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:31.731858 10843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:31.732407 10835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:31.732904 10835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:31.734061 10835 hybrid_clock.cc:648] HybridClock initialized: now 1714615171734046 us; error 23 us; skew 500 ppm
I20240502 01:59:31.735780 10835 webserver.cc:460] Webserver started at http://127.19.43.198:42979/ using document root <none> and password file <none>
I20240502 01:59:31.736272 10835 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:31.736341 10835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:31.739395 10835 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 01:59:31.741369 10849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.742354 10835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:31.742450 10835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "8fb7581a14e3402a8e7d44eef9615733"
format_stamp: "Formatted at 2024-05-02 01:59:21 on dist-test-slave-mnd3"
I20240502 01:59:31.742820 10835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:31.758978 10835 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:31.759486 10835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:31.759626 10835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:31.760098 10835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:31.760908 10835 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:31.760955 10835 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.761003 10835 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:31.761030 10835 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.769451 10835 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.198:46009
I20240502 01:59:31.769481 10965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.198:46009 every 8 connection(s)
I20240502 01:59:31.770375 10835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 01:59:31.775719 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 10835
I20240502 01:59:31.776048 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.199:46643
--local_ip_for_outbound_sockets=127.19.43.199
--tserver_master_addrs=127.19.43.254:35279
--webserver_port=46489
--webserver_interface=127.19.43.199
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:31.778596 10966 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:31.778811 10966 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:31.779304 10966 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:31.780285 8871 ts_manager.cc:194] Re-registered known tserver with Master: 8fb7581a14e3402a8e7d44eef9615733 (127.19.43.198:46009)
I20240502 01:59:31.781067 8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.198:48715
W20240502 01:59:31.883278 10969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:31.883515 10969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:31.883581 10969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:31.887692 10969 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:31.887768 10969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:31.887856 10969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.199
I20240502 01:59:31.892668 10969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.199:46643
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.19.43.199
--webserver_port=46489
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.10969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.199
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:31.893659 10969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:31.894976 10969 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:31.903009 10975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:31.902951 10974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:31.903061 10977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:31.903302 10969 server_base.cc:1034] running on GCE node
I20240502 01:59:31.903635 10969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:31.904132 10969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:31.905278 10969 hybrid_clock.cc:648] HybridClock initialized: now 1714615171905267 us; error 31 us; skew 500 ppm
I20240502 01:59:31.907148 10969 webserver.cc:460] Webserver started at http://127.19.43.199:46489/ using document root <none> and password file <none>
I20240502 01:59:31.907649 10969 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:31.907714 10969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:31.910796 10969 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20240502 01:59:31.912760 10983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.913774 10969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 01:59:31.913863 10969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "dfe1786af0dc4cc4b3149d86b0623dd9"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:31.914276 10969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:31.932087 10969 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:31.932657 10969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:31.932786 10969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:31.933252 10969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:31.934168 10969 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:31.934226 10969 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.934269 10969 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:31.934291 10969 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:31.943032 10969 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.199:46643
I20240502 01:59:31.943053 11096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.199:46643 every 8 connection(s)
I20240502 01:59:31.944069 10969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 01:59:31.951346 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 10969
I20240502 01:59:31.951725 19631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
/tmp/dist-test-taskrI6JNf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.43.200:46523
--local_ip_for_outbound_sockets=127.19.43.200
--tserver_master_addrs=127.19.43.254:35279
--webserver_port=35325
--webserver_interface=127.19.43.200
--builtin_ntp_servers=127.19.43.212:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 01:59:31.952716 11097 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:31.952960 11097 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:31.953413 11097 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:31.954368 8871 ts_manager.cc:194] Re-registered known tserver with Master: dfe1786af0dc4cc4b3149d86b0623dd9 (127.19.43.199:46643)
I20240502 01:59:31.954901 8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.199:43939
W20240502 01:59:32.062379 11100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 01:59:32.062615 11100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 01:59:32.062676 11100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 01:59:32.065999 11100 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 01:59:32.066085 11100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 01:59:32.066196 11100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.43.200
I20240502 01:59:32.069723 11100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.43.212:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.43.200:46523
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.19.43.200
--webserver_port=35325
--tserver_master_addrs=127.19.43.254:35279
--never_fsync=true
--heap_profile_path=/tmp/kudu.11100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.43.200
--log_dir=/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 01:43:14 UTC on 1a15035a6830
build id 81
I20240502 01:59:32.070541 11100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 01:59:32.071588 11100 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 01:59:32.079254 11108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:32.079331 11105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 01:59:32.079254 11106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 01:59:32.079484 11100 server_base.cc:1034] running on GCE node
I20240502 01:59:32.079862 11100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 01:59:32.080408 11100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 01:59:32.081554 11100 hybrid_clock.cc:648] HybridClock initialized: now 1714615172081541 us; error 27 us; skew 500 ppm
I20240502 01:59:32.083519 11100 webserver.cc:460] Webserver started at http://127.19.43.200:35325/ using document root <none> and password file <none>
I20240502 01:59:32.084033 11100 fs_manager.cc:359] Metadata directory not provided
I20240502 01:59:32.084098 11100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 01:59:32.087837 11100 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20240502 01:59:32.090101 11115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:32.091246 11100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 01:59:32.091356 11100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "1110b7aef73d4823bb3fa069b92a1279"
format_stamp: "Formatted at 2024-05-02 01:59:22 on dist-test-slave-mnd3"
I20240502 01:59:32.091842 11100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 01:59:32.105281 11100 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 01:59:32.105825 11100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 01:59:32.105978 11100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 01:59:32.106495 11100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 01:59:32.107368 11100 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 01:59:32.107415 11100 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:32.107467 11100 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 01:59:32.107496 11100 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 01:59:32.115917 11100 rpc_server.cc:307] RPC server started. Bound to: 127.19.43.200:46523
I20240502 01:59:32.115932 11228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.43.200:46523 every 8 connection(s)
I20240502 01:59:32.116933 11100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrI6JNf/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714615075986586-19631-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 01:59:32.125389 11229 heartbeater.cc:347] Connected to a master server at 127.19.43.254:35279
I20240502 01:59:32.125602 11229 heartbeater.cc:464] Registering TS with master...
I20240502 01:59:32.126060 11229 heartbeater.cc:511] Master 127.19.43.254:35279 requested a full tablet report, sending...
I20240502 01:59:32.126933 8871 ts_manager.cc:194] Re-registered known tserver with Master: 1110b7aef73d4823bb3fa069b92a1279 (127.19.43.200:46523)
I20240502 01:59:32.127215 19631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrI6JNf/build/debug/bin/kudu as pid 11100
I20240502 01:59:32.127419 19631 test_util.cc:268] Using random seed: -196945221
I20240502 01:59:32.127533 8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.43.200:42923
I20240502 01:59:32.137984 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240502 01:59:32.138238 19631 test_util.cc:268] Using random seed: -196934405
I20240502 01:59:32.148025 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240502 01:59:32.148187 19631 test_util.cc:268] Using random seed: -196924454
I20240502 01:59:32.160876 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240502 01:59:32.161031 19631 test_util.cc:268] Using random seed: -196911611
I20240502 01:59:32.164808 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.164984 9157 raft_consensus.cc:465] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 01:59:32.167227 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.167348 9158 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.167402 9158 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.168453 9158 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.168816 9158 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.169049 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.169164 9294 raft_consensus.cc:3032] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.169202 9294 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.169340 9294 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.169608 9294 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.169586 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.169706 9022 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.170584 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.170634 9294 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.170672 9022 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.170696 9157 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.170749 9157 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.171011 9089 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.171685 10553 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.171664 9157 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.171865 10553 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.171986 9157 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.172804 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.172837 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.172919 9022 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.172919 9157 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.171994 10553 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.172968 9157 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.173027 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.173136 9426 raft_consensus.cc:3032] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.173175 9426 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.173321 9426 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.173619 9426 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.173727 9022 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.173729 9157 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.174049 9090 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.174283 10553 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.174358 10553 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.174464 10553 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.174645 9426 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.176479 8872 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.177934 9157 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.178020 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.178191 9022 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.178233 9022 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.178408 9022 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.178665 9022 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.179651 9022 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.180467 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.180578 9293 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.180831 9090 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.181077 10553 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.181160 10553 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.181273 10553 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.181543 9293 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.181962 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.182085 9157 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.182134 9157 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.183008 9157 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.184098 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.184230 9425 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.184367 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.184481 9022 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.184527 9022 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.184659 9022 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.184831 9022 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.185169 9425 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.185451 9089 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.185663 9022 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.185683 11289 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.185987 9157 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.186391 11289 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.186625 11289 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.186847 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.186957 9158 raft_consensus.cc:465] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 01:59:32.188071 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.188176 9158 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.188236 9158 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.189093 9158 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.189375 9158 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.190989 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.191028 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.191088 9158 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.191135 9158 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.191155 9294 raft_consensus.cc:3032] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.191200 9294 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.191331 9294 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.191491 9294 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.191987 9158 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.192306 9158 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.192441 9294 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.192545 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.192652 9021 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.192688 9021 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.192822 9021 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.192986 9021 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.193337 9089 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.193573 10553 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.193665 10553 raft_consensus.cc:686] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.193754 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.193775 10553 consensus_queue.cc:237] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.193859 9423 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.193946 9021 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.194330 9090 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.194475 11288 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.194542 11288 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.194645 11288 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.194872 9423 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.196099 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.196214 9158 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.196280 9158 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:32.196760 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34072: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.197117 9158 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.197429 9158 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.198161 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.198266 9294 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.198297 9294 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.198398 9294 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.198549 9294 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.198705 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.198725 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.198825 9021 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.198814 9425 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.199457 9294 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.199728 9021 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.199790 9425 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.199990 9090 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.200251 10553 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.200340 10553 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.200342 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.200453 9158 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.200500 9158 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.200464 10553 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:32.200256 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60690: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.201347 9158 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.201649 9158 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.202638 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.202651 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.202744 9425 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.202744 9293 raft_consensus.cc:3032] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.202780 9293 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.202903 9293 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.203049 9293 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.203624 9425 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.203894 9293 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.203955 9089 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.204128 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.204223 9158 raft_consensus.cc:465] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Not starting forced leader election -- already a leader
W20240502 01:59:32.204464 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56722: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.204849 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.204952 9158 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.205003 9158 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:32.205715 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60704: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.205765 9158 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.206118 9158 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.206547 11270 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.206673 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.206810 9021 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.206789 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.206748 11270 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.206892 9293 raft_consensus.cc:3032] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.206938 9293 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.207079 9293 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.207146 11270 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.207279 9293 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.207969 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.208066 9158 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.208114 9158 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.208098 9022 raft_consensus.cc:1261] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.208215 9293 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.208467 11289 consensus_queue.cc:1035] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.208768 9089 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.208997 9158 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.209084 9425 raft_consensus.cc:1261] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.209301 9158 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.211409 11270 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.211495 11270 raft_consensus.cc:686] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.211606 11270 consensus_queue.cc:237] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.211866 11288 consensus_queue.cc:1035] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.211812 8871 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.212788 9294 raft_consensus.cc:1261] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.213153 9425 raft_consensus.cc:1261] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.213157 10553 consensus_queue.cc:1035] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.213294 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.213387 9293 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.213986 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.214119 9424 raft_consensus.cc:3032] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.214161 9424 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.214283 9424 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.214401 9293 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.214426 9424 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.215292 9424 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.216153 9089 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.216287 11270 consensus_queue.cc:1035] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.216529 11270 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.216607 11270 raft_consensus.cc:686] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.216727 11270 consensus_queue.cc:237] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.218459 8875 catalog_manager.cc:5484] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.220381 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.220504 9158 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.220566 9158 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.220902 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240502 01:59:32.221069 19631 test_util.cc:268] Using random seed: -196851573
I20240502 01:59:32.221549 9158 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.222061 9158 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.222234 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.222348 9424 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.222718 9021 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.222826 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.222918 9020 raft_consensus.cc:1261] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.222925 9023 raft_consensus.cc:3032] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.222996 9023 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.223146 9023 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.223160 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.223277 9158 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.223307 9023 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.223328 9158 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.223701 9424 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.224167 9023 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.224165 9158 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.224252 9089 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.224409 11289 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.224469 11289 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.224582 11289 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.225312 11270 consensus_queue.cc:1035] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.225468 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.225567 9023 raft_consensus.cc:3032] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.225601 9023 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.225736 9023 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.225898 9023 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.226626 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.226747 9294 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.227815 9294 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.228068 9023 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.228446 9090 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.228633 11270 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.228709 11270 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.228796 9293 raft_consensus.cc:1261] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.228816 11270 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.229171 11288 consensus_queue.cc:1035] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.231370 9158 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.233192 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.233310 9157 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.233361 9157 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.234334 9157 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.234666 9157 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.235605 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.235728 9021 raft_consensus.cc:3032] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.235772 9021 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.235905 9021 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.236088 9021 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.236383 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.236505 9426 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.237067 9021 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.237376 9090 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.237443 9426 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.237639 10553 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.237715 10553 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.237841 10553 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.238606 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.238734 9157 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.238797 9157 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.239656 9157 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.239966 9157 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.240870 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.240980 9422 raft_consensus.cc:3032] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.241019 9422 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.241165 9422 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.241196 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.241309 9157 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.241340 9422 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.241361 9157 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.242300 9422 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.242538 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.242556 9157 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.242664 9019 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.242959 9157 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.243146 11268 mvcc.cc:205] Tried to move back new op lower bound from 7023063745419485184 to 7023063745334267904. Current Snapshot: MvccSnapshot[applied={T|T < 7023063745419485184}]
I20240502 01:59:32.243608 9019 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.243803 9089 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.243834 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.243937 9424 raft_consensus.cc:3032] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.243952 11289 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.243986 9424 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.244020 11289 raft_consensus.cc:686] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.244132 11289 consensus_queue.cc:237] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.244113 9424 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.244293 9424 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.245199 9424 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
W20240502 01:59:32.245337 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34072: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.245980 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.246155 9157 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.246251 9157 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.247102 9157 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.247968 9157 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.248185 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.248308 9292 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.248363 9292 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.248507 9292 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.248699 9292 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.249616 9292 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
W20240502 01:59:32.249775 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56722: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.250330 9089 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.250475 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.250614 9157 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.250701 10553 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.250748 9157 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.250794 10553 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.250957 10553 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.252164 9157 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.252620 9157 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.252905 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.253018 9292 raft_consensus.cc:3032] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.253069 9292 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.253196 9292 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.253306 9292 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.254319 9292 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.254412 9089 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.254647 11289 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.254737 11289 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.254866 11289 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.255138 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.255278 9157 raft_consensus.cc:465] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Not starting forced leader election -- already a leader
W20240502 01:59:32.255474 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56732: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.256162 9089 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.256300 10553 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.256366 10553 raft_consensus.cc:686] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.256466 10553 consensus_queue.cc:237] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.256084 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.257313 9157 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.257418 9157 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.258384 9018 raft_consensus.cc:1261] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 01:59:32.258425 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.258389 9157 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.258504 9020 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.259229 11270 consensus_queue.cc:1035] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:32.259464 9020 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.259848 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.259958 9019 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.260391 9423 raft_consensus.cc:1261] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 01:59:32.260852 9019 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.260994 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.261107 9292 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.261162 9292 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.261301 9292 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.261482 9292 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.261554 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.261649 9021 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.262420 9292 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.262398 11289 consensus_queue.cc:1035] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:32.262542 9021 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.262952 9089 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.263132 9157 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.263350 11289 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.263435 11289 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.263563 11289 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.265385 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.265502 9424 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.266480 9424 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 2.
I20240502 01:59:32.269248 9021 raft_consensus.cc:1261] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.269558 9422 raft_consensus.cc:1261] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.270574 11270 consensus_queue.cc:1035] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.271492 11270 consensus_queue.cc:1035] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:59:32.272926 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34082: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.280332 11256 mvcc.cc:205] Tried to move back new op lower bound from 7023063745609965568 to 7023063745513259008. Current Snapshot: MvccSnapshot[applied={T|T < 7023063745609965568}]
I20240502 01:59:32.287209 8871 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.287487 8871 catalog_manager.cc:5484] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 1 to 2, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.293694 8875 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.293977 8875 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.294241 8875 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.294483 9422 raft_consensus.cc:1261] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.294458 8875 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.294622 8875 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.294800 8875 catalog_manager.cc:5484] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.295446 9294 raft_consensus.cc:1261] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.295557 11289 consensus_queue.cc:1035] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.298537 10553 consensus_queue.cc:1035] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.300936 11260 mvcc.cc:205] Tried to move back new op lower bound from 7023063745609965568 to 7023063745513259008. Current Snapshot: MvccSnapshot[applied={T|T < 7023063745609965568}]
W20240502 01:59:32.300979 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34072: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.302346 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240502 01:59:32.302520 19631 test_util.cc:268] Using random seed: -196770124
I20240502 01:59:32.316562 8875 catalog_manager.cc:5484] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.316875 8875 catalog_manager.cc:5484] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.317613 8875 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.319211 8875 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.319419 8875 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.319576 8875 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.319754 8875 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 2 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.331898 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.332065 9023 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.332137 9023 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.333207 9023 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.333572 9023 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.334340 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.334457 9292 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.335090 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.335202 9426 raft_consensus.cc:3032] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.335242 9426 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.335382 9426 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.335433 9292 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.335546 9426 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.336509 9426 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.337000 8956 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.337427 11293 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.337508 11293 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.337635 11293 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.337743 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.337847 9020 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.337894 9020 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.338824 9020 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.339164 9020 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.340217 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.340332 9158 raft_consensus.cc:3032] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.340378 9158 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.340515 9158 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.340766 9158 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.340996 8878 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.341774 9158 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.342644 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.342753 9292 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.343653 9292 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.344259 8956 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.345209 11264 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.345278 11264 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.345387 11264 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.346370 9021 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.346477 9021 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.346527 9021 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.347463 9021 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.347760 9021 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.349092 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.349210 9421 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.349352 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.349473 9294 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.349527 9294 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.349646 9294 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.349794 9294 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.350224 9421 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.350651 9294 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.352334 8956 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.352622 10181 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.352684 10181 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.352777 10181 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.355638 9021 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.355747 9021 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.355798 9021 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.355943 8878 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.356767 9021 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.357280 9021 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.357431 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.357534 9158 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.357571 9158 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.357690 9158 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.357837 9158 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.358824 9158 raft_consensus.cc:2396] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.361805 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.361924 9020 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.361979 9020 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.362942 9020 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.362993 8878 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.363263 9020 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.364413 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.364523 9292 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.365196 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.365312 9426 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.365489 9292 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.365594 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.365694 9158 raft_consensus.cc:3032] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.365732 9158 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.365860 9158 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.365957 9158 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.366245 9426 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.366966 9158 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 2 because replica has last-logged OpId of term: 1 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:32.367440 8956 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.367597 11264 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.367691 11264 raft_consensus.cc:686] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.367812 11264 consensus_queue.cc:237] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.369014 8956 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.369139 11293 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.369196 11293 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.369302 11293 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.369953 8878 catalog_manager.cc:5484] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.370065 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.370158 9018 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.370201 9018 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.371023 9018 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.371323 9018 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.371688 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.371798 9293 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.372499 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.372602 9158 raft_consensus.cc:3032] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.372656 9158 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.372666 9293 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.372773 9158 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.372916 9158 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.373095 8956 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.373250 11293 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.373338 11293 raft_consensus.cc:686] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.373461 11293 consensus_queue.cc:237] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.373833 9158 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.374511 9020 raft_consensus.cc:1226] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Rejecting Update request from peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 for earlier term 1. Current term is 2. Ops: [1.8-1.8]
I20240502 01:59:32.377128 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.377236 9023 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.377296 9023 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:32.379241 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60716: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.379230 9023 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.379524 9023 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.379448 8878 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.379679 8878 catalog_manager.cc:5484] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.381094 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.381209 9426 raft_consensus.cc:3032] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.381225 9021 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.381249 9426 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.381462 9426 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.381527 11270 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.381645 9426 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.384111 9426 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.386497 9424 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.387074 11288 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.387272 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.387389 9293 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.388298 9293 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.391849 8956 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.392005 11264 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.392076 11264 raft_consensus.cc:686] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.392194 11264 consensus_queue.cc:237] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.394269 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.394289 8878 catalog_manager.cc:5484] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.394398 9020 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.394457 9020 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.395259 9020 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.395802 9020 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.396350 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.396464 9158 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.396507 9158 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.396633 9158 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.396813 9158 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.397138 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.397238 9293 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.397727 9158 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.397989 8957 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.398169 9293 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.398223 10181 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.398304 10181 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.398416 10181 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.399080 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.399226 9020 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.399273 9020 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:32.399152 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56768: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.400152 9020 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.400385 8878 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.400439 9020 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.400854 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.400939 9158 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.400972 9158 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.401077 9158 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.401203 9158 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.401953 9158 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.404475 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.404584 9426 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.405620 9426 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.411769 9293 raft_consensus.cc:1261] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 01:59:32.412225 9020 raft_consensus.cc:1261] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 01:59:32.412257 11288 consensus_queue.cc:1035] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:32.412771 11340 consensus_queue.cc:1035] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 01:59:32.419479 8956 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.420056 10181 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.420133 10181 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.420251 10181 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.421890 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.421988 9023 raft_consensus.cc:465] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 01:59:32.422559 8875 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.429003 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.429121 9020 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.429188 9020 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.430236 9020 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.430675 9020 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.433987 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.434113 9422 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.434157 9020 raft_consensus.cc:1226] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Rejecting Update request from peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 for earlier term 2. Current term is 3. Ops: []
I20240502 01:59:32.434959 9422 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.435503 8956 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.436094 10181 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.436173 10181 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.436297 10181 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.436707 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.436820 9158 raft_consensus.cc:3032] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.436858 9158 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.437052 9158 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.437608 9422 raft_consensus.cc:1226] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Rejecting Update request from peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 for earlier term 2. Current term is 3. Ops: [2.9-2.9]
I20240502 01:59:32.438441 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.438544 9018 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.438591 9018 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.438997 9158 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.439541 9018 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.439862 9018 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.440127 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.440218 9157 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.440253 9157 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.440294 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.440393 9293 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.440364 9157 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.440485 9157 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.441428 9157 raft_consensus.cc:2396] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.441450 9293 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.442234 8956 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.442329 8875 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.442445 11352 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.442696 11352 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.442898 11352 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.443408 9158 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:32.449142 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.449244 9018 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.449298 9018 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.450172 9018 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.450479 9018 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.450978 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.451131 9293 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.451812 8871 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.452000 9293 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.452510 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_5 already exists
I20240502 01:59:32.452553 8956 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.452657 19631 test_util.cc:268] Using random seed: -196619984
I20240502 01:59:32.452697 11352 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.452757 11352 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.452864 11352 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.454360 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.454459 9018 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.454505 9018 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.454170 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.454906 9158 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.454993 9158 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.455189 9158 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.455304 9018 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.455379 9158 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.455600 9018 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.456454 9158 raft_consensus.cc:2396] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.457532 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.457641 9293 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.457685 9293 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.457823 9293 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.457998 9293 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.458861 9293 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.460065 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.460191 9424 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
W20240502 01:59:32.460850 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.461138 9424 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.461930 8956 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.462193 11352 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.462270 11352 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.462396 11352 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.464767 8878 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.466720 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
W20240502 01:59:32.467316 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43440: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.466879 9020 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.469041 9020 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:32.470458 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34090: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.471072 9020 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.471318 9018 raft_consensus.cc:1226] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Rejecting Update request from peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 for earlier term 2. Current term is 3. Ops: [2.8-2.8]
I20240502 01:59:32.471800 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.471906 9292 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.471978 11288 consensus_queue.cc:1046] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: INVALID_TERM, Last received: 2.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:32.472246 9020 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.472292 11270 raft_consensus.cc:3032] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.472337 11270 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.472453 11270 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.472595 11270 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.472860 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "877896deb7564195b8683801b048c80c"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.472893 9292 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 01:59:32.472949 9020 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.472993 9020 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.473299 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.473930 9158 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 01:59:32.474200 9020 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.474218 8957 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
W20240502 01:59:32.474314 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60690: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.474435 11293 raft_consensus.cc:2726] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 01:59:32.474864 9020 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.474853 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.474947 9292 raft_consensus.cc:3032] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.474982 9292 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.475028 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.475107 9292 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.475149 9426 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.475272 9292 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.476068 9292 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.476068 9426 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.476323 8956 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.476483 11293 raft_consensus.cc:2781] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.476565 11293 raft_consensus.cc:686] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.476692 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.476696 11293 consensus_queue.cc:237] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.476848 9020 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.476905 9020 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.476393 8875 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.478356 9020 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.479218 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.479326 9426 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.479522 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.479624 9292 raft_consensus.cc:3032] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.479710 9292 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.479983 9292 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.480161 9292 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.480166 9426 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.480448 8956 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.480612 11293 raft_consensus.cc:2781] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.480686 11293 raft_consensus.cc:686] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.480804 11293 consensus_queue.cc:237] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.481470 9292 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 2.
I20240502 01:59:32.481521 9020 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 2 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.482949 9158 raft_consensus.cc:1261] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20240502 01:59:32.483425 10181 consensus_queue.cc:1035] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 01:59:32.483707 8878 catalog_manager.cc:5484] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.483870 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.483965 9020 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.484017 9020 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.483943 8878 catalog_manager.cc:5484] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 1 to 2, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 2 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.485059 9020 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.485173 9292 raft_consensus.cc:1261] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20240502 01:59:32.486353 9292 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.486404 10181 consensus_queue.cc:1035] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 01:59:32.487397 10181 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.489048 9426 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.491077 10181 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.492122 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.492231 9422 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.493099 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_6 already exists
I20240502 01:59:32.493225 9422 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.493283 19631 test_util.cc:268] Using random seed: -196579358
I20240502 01:59:32.494230 8956 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.494411 11293 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.494451 9020 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.494498 11293 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.494612 11293 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.495296 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.495404 9020 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.495453 9020 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.496373 9020 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.496814 8878 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.496933 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.497035 9294 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.497711 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.497821 9157 raft_consensus.cc:3032] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.497861 9157 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.497923 9294 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.497980 9157 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.498139 9157 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.498643 8956 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.498980 10181 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.499053 10181 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.499147 9157 raft_consensus.cc:2396] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.499153 10181 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.500113 9020 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.500677 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.500785 9018 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.500833 9018 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.501711 9018 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.502060 9018 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.502557 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.502667 9422 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.502735 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.502831 9157 raft_consensus.cc:3032] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.502867 9157 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.502987 9157 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.503118 9157 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.503600 9422 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.503957 9157 raft_consensus.cc:2396] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.504235 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.504339 9018 raft_consensus.cc:465] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 01:59:32.504575 8956 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.505112 11293 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.505087 8878 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.505208 11293 raft_consensus.cc:686] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.505316 11293 consensus_queue.cc:237] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.506387 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.506484 9018 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.506532 9018 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.506644 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.506737 9157 raft_consensus.cc:3032] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.506770 9157 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.506887 9157 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.507033 9157 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.507406 9018 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.507665 9157 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.507761 9018 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.508143 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.508275 9422 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.509131 9422 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.509637 8956 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.509764 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:32.509873 9018 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.509932 9018 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.510062 11264 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.510155 11264 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.510295 11264 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.510766 9018 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.510998 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.511073 9018 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.511092 9157 raft_consensus.cc:3032] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.511128 9157 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.511267 9157 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.511431 9157 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.512151 9157 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.512934 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.513038 9422 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.513237 8878 catalog_manager.cc:5484] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.513887 9422 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.514312 8956 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.514528 11293 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.514601 11293 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.514708 11293 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.515091 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.515411 9157 raft_consensus.cc:3032] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.515453 9157 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.515866 9157 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.516031 9157 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.522642 8871 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.522876 8871 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 3 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.525573 9424 raft_consensus.cc:1261] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.525681 9157 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 3.
I20240502 01:59:32.526346 11264 consensus_queue.cc:1035] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.527926 9294 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.528914 11288 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 01:59:32.528973 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43440: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.529302 9426 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.529711 9294 raft_consensus.cc:1261] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 01:59:32.530644 9158 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.530720 11288 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.531368 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_7 already exists
I20240502 01:59:32.531517 19631 test_util.cc:268] Using random seed: -196541124
I20240502 01:59:32.532549 11293 consensus_queue.cc:1035] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.533022 9292 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.533068 11352 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.533174 11352 consensus_queue.cc:1225] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:32.533936 9158 pending_rounds.cc:76] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting all ops after (but not including) 3
I20240502 01:59:32.533993 9158 pending_rounds.cc:97] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 01:59:32.534096 9158 raft_consensus.cc:2866] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:32.535444 11264 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.542901 9291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
W20240502 01:59:32.543017 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34096: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.543051 9291 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.543112 9291 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.544068 9291 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.544394 9291 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.545419 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.545538 9424 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.546569 9424 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.546914 9226 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.547077 10622 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.547178 10622 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.547317 10622 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.549062 9291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.549180 9291 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.549230 9291 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.550437 9292 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.550796 11352 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.552255 8875 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.552693 9158 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.553000 9424 raft_consensus.cc:1261] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20240502 01:59:32.553356 9157 raft_consensus.cc:1261] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20240502 01:59:32.553412 11293 consensus_queue.cc:1035] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:32.553551 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.553654 9020 raft_consensus.cc:3032] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.553699 9020 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.553779 11293 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.553814 9020 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.553881 11293 consensus_queue.cc:1225] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:32.553943 9020 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:32.553995 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.554587 11352 consensus_queue.cc:1035] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:32.554790 9291 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.555342 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.555444 9157 raft_consensus.cc:3032] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.555480 9157 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.555600 9157 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.555763 9157 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.556710 9157 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.558374 9227 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.558403 9157 pending_rounds.cc:76] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting all ops after (but not including) 3
I20240502 01:59:32.558445 9157 pending_rounds.cc:97] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 01:59:32.558473 9157 raft_consensus.cc:2866] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:32.558539 10535 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.558614 10535 raft_consensus.cc:686] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.558727 10535 consensus_queue.cc:237] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.560163 9426 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 01:59:32.560842 10181 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:32.562270 9156 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 01:59:32.563761 11264 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:32.575065 9291 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.576053 9020 raft_consensus.cc:2396] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.578405 9292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.578527 9292 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.578585 9292 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.579603 9292 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.579972 9292 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.581321 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.581491 9020 raft_consensus.cc:3032] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.581583 9020 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.581773 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.581874 9293 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.581923 9293 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.582535 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.582643 9157 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.582892 9293 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.583225 9293 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.583614 9157 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.583837 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.583936 9018 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.583969 9018 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.584103 9018 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.584232 9018 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.584769 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.584882 9426 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.585189 9018 raft_consensus.cc:2396] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.585788 9426 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.586925 8875 catalog_manager.cc:5484] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 1 to 2, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 2 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.587379 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.587489 9422 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.588387 9422 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.588831 9226 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:32.589138 10622 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.589216 10622 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.589326 10622 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.590167 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.590268 9293 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.590314 9293 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.590920 9293 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.591166 9293 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.591761 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.591871 9293 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.591921 9293 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.592000 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.592109 9018 raft_consensus.cc:3032] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.592155 9018 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.592294 9018 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.592422 9018 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.592716 9293 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.593017 9293 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.593305 9018 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:32.594343 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.594462 9293 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.594511 9293 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.594799 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.594906 9158 raft_consensus.cc:3032] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.594940 9158 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.595054 9158 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.595206 9158 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.595284 9293 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.595566 9293 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.594019 8875 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.596138 9158 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.596467 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.596560 9293 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.596604 9293 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.597301 9227 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.597362 9293 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.597453 10535 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.597515 10535 raft_consensus.cc:686] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.597616 10535 consensus_queue.cc:237] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.597821 9293 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.598013 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.598147 9424 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.598392 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.598510 9022 raft_consensus.cc:3032] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.598554 9022 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.598676 9022 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.598722 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.598811 9293 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.598836 9022 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.599021 9293 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.599109 9424 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.599428 9226 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.599625 10535 raft_consensus.cc:2781] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.599705 10535 raft_consensus.cc:686] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.599767 9022 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.599809 10535 consensus_queue.cc:237] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.600054 9293 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.600574 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.600674 9157 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.601161 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.601255 9426 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.601595 9157 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 1 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:32.601660 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.601740 9156 raft_consensus.cc:3032] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.601778 9156 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.601893 9156 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.602062 9156 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.602156 9426 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.602217 9227 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.602483 10535 raft_consensus.cc:2726] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 01:59:32.602488 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.602519 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.602577 9422 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.602603 9158 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.602988 9156 raft_consensus.cc:2396] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.603161 9422 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.603374 9158 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 4.
I20240502 01:59:32.603411 9226 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.603435 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.603523 9018 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.603564 9018 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.603566 10535 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.603628 10535 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.603708 9018 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 3.16, Last appended by leader: 16, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.603727 10535 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.603808 9018 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.603853 9293 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.604657 9018 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 4.
I20240502 01:59:32.604735 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.604843 9293 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.604885 9293 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.605183 8871 catalog_manager.cc:5484] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.605558 9293 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.606112 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.606199 9422 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.606773 9291 raft_consensus.cc:1226] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 3. Current term is 4. Ops: []
I20240502 01:59:32.606946 9422 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.607131 9293 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.607300 9226 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.607450 10622 raft_consensus.cc:2781] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.607518 10622 raft_consensus.cc:686] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.607617 10622 consensus_queue.cc:237] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.607702 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.607801 9293 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.607848 9293 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.608536 9293 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.608197 9020 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.608829 9293 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.609395 9020 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.609416 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.609510 9422 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.610108 9020 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.610340 9422 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.610656 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.610767 9293 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.610795 9226 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.610828 9293 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.611045 10622 raft_consensus.cc:2781] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.611126 10622 raft_consensus.cc:686] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.611232 10622 consensus_queue.cc:237] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.611801 9293 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.612210 9227 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.612381 9227 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.612370 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.612460 9422 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.612941 10622 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.613127 10535 raft_consensus.cc:2726] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.613158 10622 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.613327 9422 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.613298 10622 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.16, Last appended by leader: 16, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.613473 9293 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.613571 9226 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.613749 11419 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.613895 11419 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.614107 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.614226 9156 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.614269 9156 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.614252 9292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "877896deb7564195b8683801b048c80c"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.614346 9292 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.614391 9292 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.614403 9156 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.614562 9156 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.614984 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.615110 9158 raft_consensus.cc:3032] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.615149 9158 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.615185 9292 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.615293 9158 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.615411 9156 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.615444 9158 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.615504 9292 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.615816 11419 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.615867 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.615957 9426 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.616369 9158 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.616837 9426 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.616925 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.617045 9018 raft_consensus.cc:3032] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.617082 9018 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.617198 9226 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.617218 9018 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.617350 9018 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.617559 8871 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.617797 8871 catalog_manager.cc:5484] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 2 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.617947 8871 catalog_manager.cc:5484] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 1 to 2, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 2 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.618157 8871 catalog_manager.cc:5484] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 1 to 2, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 2 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.618337 9018 raft_consensus.cc:2396] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.618680 10832 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:32.622105 11419 raft_consensus.cc:2781] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.622090 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.622220 11419 raft_consensus.cc:686] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.622351 11419 consensus_queue.cc:237] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.624401 9425 raft_consensus.cc:1226] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 1. Current term is 2. Ops: [1.13-1.13]
I20240502 01:59:32.625362 9422 raft_consensus.cc:1261] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.625329 10181 consensus_queue.cc:1046] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: INVALID_TERM, Last received: 1.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 01:59:32.625820 10181 raft_consensus.cc:3032] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.625885 10181 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.626008 10181 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.13, Last appended by leader: 13, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.626179 10181 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.626892 9293 raft_consensus.cc:1226] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 1. Current term is 2. Ops: [1.13-1.13]
I20240502 01:59:32.627318 10181 consensus_queue.cc:1035] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.630542 9292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.630683 9292 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.630746 9292 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.631747 9292 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.632084 9292 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.632953 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
W20240502 01:59:32.633023 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.633070 9424 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.633888 8873 catalog_manager.cc:5484] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.634099 9424 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.634136 8873 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 3 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 4 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.634332 8873 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.635919 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.635984 9226 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.636047 9020 raft_consensus.cc:3032] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.636092 9020 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.636271 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.636337 10622 raft_consensus.cc:2781] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.636384 9293 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.636392 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_8 already exists
I20240502 01:59:32.636416 10622 raft_consensus.cc:686] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.636435 9293 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.636554 19631 test_util.cc:268] Using random seed: -196436087
I20240502 01:59:32.636531 10622 consensus_queue.cc:237] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.637349 9293 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.637979 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.637923 9020 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.638113 9018 raft_consensus.cc:3032] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.638149 9018 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.638151 9020 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.638284 9018 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.638423 9018 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.639170 9158 raft_consensus.cc:1261] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.639267 9018 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 4.
I20240502 01:59:32.639710 11413 consensus_queue.cc:1035] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.639892 11413 consensus_queue.cc:1225] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:32.640651 8873 catalog_manager.cc:5484] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.641204 9227 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.641362 11419 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.641438 11419 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.641546 11419 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.642277 9293 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.643570 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.643678 9158 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.644054 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.644657 9020 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:32.644649 9158 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 4.
I20240502 01:59:32.645048 9424 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.645334 8873 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 3 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 4 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.645427 10622 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.646131 9020 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.647061 10622 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.647202 10622 consensus_queue.cc:1225] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:32.647064 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.647521 9158 pending_rounds.cc:76] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting all ops after (but not including) 3
I20240502 01:59:32.647564 9158 pending_rounds.cc:97] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 01:59:32.647599 9158 raft_consensus.cc:2866] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:32.647716 9293 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.647840 9293 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.649724 9293 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.650107 9293 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.655059 11414 mvcc.cc:205] Tried to move back new op lower bound from 7023063747056771072 to 7023063746234933248. Current Snapshot: MvccSnapshot[applied={T|T < 7023063747056771072 or (T in {7023063747056771072})}]
I20240502 01:59:32.655404 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.655612 9293 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.655733 9293 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:32.656678 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.656833 9422 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 01:59:32.657172 9020 pending_rounds.cc:76] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: Aborting all ops after (but not including) 3
I20240502 01:59:32.657239 9020 pending_rounds.cc:97] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 01:59:32.657240 10535 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:32.657325 9020 raft_consensus.cc:2866] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:32.659456 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.659572 9018 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.659617 9018 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.659751 9018 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.659907 9018 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.660282 9293 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.660915 9018 raft_consensus.cc:2396] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.660951 9424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.661053 9424 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.661720 9157 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 01:59:32.661996 9424 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.662513 9293 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.662861 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.662973 9156 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.663924 9156 raft_consensus.cc:2396] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.664254 9292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.664943 9292 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.665017 9292 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.665931 9292 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.666268 9292 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.667742 9424 raft_consensus.cc:1261] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.668079 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.668183 9293 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.668213 9226 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.668232 11264 consensus_queue.cc:1035] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.668694 9425 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.668912 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.669010 9018 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.669580 10535 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.669649 10535 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.669759 10535 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.669883 9018 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 4.
I20240502 01:59:32.670696 9227 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.670892 9227 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.671320 10622 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.671353 10535 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:32.671396 10622 raft_consensus.cc:686] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.671490 10622 consensus_queue.cc:237] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.671576 10535 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.671840 10535 raft_consensus.cc:2726] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.668236 9293 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.673297 11341 mvcc.cc:205] Tried to move back new op lower bound from 7023063747149836288 to 7023063746927230976. Current Snapshot: MvccSnapshot[applied={T|T < 7023063747149836288}]
I20240502 01:59:32.674178 9293 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.674798 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.674903 9425 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.675406 9155 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.675733 9154 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 01:59:32.675791 9018 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 01:59:32.675827 9425 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.675398 9158 raft_consensus.cc:1261] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 01:59:32.676934 8873 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.677100 9226 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.677361 10622 raft_consensus.cc:2781] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 01:59:32.677455 10622 raft_consensus.cc:686] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.677563 10622 consensus_queue.cc:237] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.675383 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.677695 9157 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.678431 9293 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 2 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.678691 9157 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 4.
I20240502 01:59:32.679697 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.679806 9020 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.679849 9020 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.679978 9020 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.680410 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.680562 9293 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.680677 9293 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.681023 11264 consensus_queue.cc:1035] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.681123 11264 consensus_queue.cc:1225] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:32.682160 9293 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.682605 9293 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.683970 9020 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.684088 9153 pending_rounds.cc:76] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting all ops after (but not including) 3
I20240502 01:59:32.684130 9153 pending_rounds.cc:97] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 01:59:32.684160 9153 raft_consensus.cc:2866] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:32.685137 9020 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.686637 9292 raft_consensus.cc:1226] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20240502 01:59:32.687743 11293 consensus_queue.cc:1046] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:32.687942 11352 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.688005 11352 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.688143 11352 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.688315 11352 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.688568 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.688673 9153 raft_consensus.cc:3032] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 01:59:32.688709 9153 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.688833 9153 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.688998 9153 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 01:59:32.689872 9153 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 2.
I20240502 01:59:32.690120 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.690210 9157 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.690691 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.690722 11431 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:32.690830 11431 consensus_queue.cc:1225] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:32.690974 9157 raft_consensus.cc:2396] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 01:59:32.691080 11431 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:32.692085 9156 pending_rounds.cc:76] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting all ops after (but not including) 3
I20240502 01:59:32.692135 9156 pending_rounds.cc:97] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 01:59:32.692174 9156 raft_consensus.cc:2866] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:32.692373 11430 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:32.692839 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.692955 9293 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.693009 9293 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.694018 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.694607 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.694962 19631 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_9 already exists
I20240502 01:59:32.695056 9020 raft_consensus.cc:2396] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 01:59:32.695772 9293 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.696053 9293 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.696028 9227 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.696543 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.696633 9018 raft_consensus.cc:3032] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.696666 9018 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.696782 9018 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.696894 9018 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.698228 9293 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.698346 9293 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.698398 9293 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.698663 9018 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.698730 8871 catalog_manager.cc:5484] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 4, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 4 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.699007 8871 catalog_manager.cc:5484] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 1 to 2, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 2 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.699440 9293 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.699527 10535 raft_consensus.cc:2726] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.699784 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.699887 9153 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.700793 9153 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.701524 9425 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 01:59:32.702086 10622 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:32.703087 11271 mvcc.cc:205] Tried to move back new op lower bound from 7023063747222523904 to 7023063746985922560. Current Snapshot: MvccSnapshot[applied={T|T < 7023063747222523904 or (T in {7023063747222523904})}]
I20240502 01:59:32.703457 9293 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.704041 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.704146 9421 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.705066 9421 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.705588 9226 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.706305 9157 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 01:59:32.706809 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.706907 9157 raft_consensus.cc:3032] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Stepping down as leader of term 2
W20240502 01:59:32.706909 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.706941 9157 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.707063 9157 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.707242 9157 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.707815 10535 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.707891 10535 raft_consensus.cc:686] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.708000 10535 consensus_queue.cc:237] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.708135 9157 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.710919 9227 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.711354 10622 raft_consensus.cc:2726] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.713383 10622 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:32.714216 9018 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 01:59:32.717162 9425 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 01:59:32.718166 10622 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:32.722704 9292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:32.722870 9292 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.722992 9292 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.724902 9292 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.725399 9292 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.725764 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.725870 9421 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.726994 9421 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 3.
I20240502 01:59:32.730211 8873 catalog_manager.cc:5484] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.730619 9226 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.730868 10535 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.730937 10535 raft_consensus.cc:686] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.731065 10535 consensus_queue.cc:237] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.734225 10535 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:32.740803 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.741109 9020 raft_consensus.cc:3032] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.741199 9020 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.741428 9020 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.741683 9020 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.741865 9425 raft_consensus.cc:1226] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 2. Current term is 3. Ops: [2.12-2.12]
I20240502 01:59:32.743043 9020 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 3 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 01:59:32.743438 9292 raft_consensus.cc:1226] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 2. Current term is 3. Ops: [2.12-2.12]
W20240502 01:59:32.749486 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:32.750002 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:32.753042 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.753525 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.753765 9422 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.753830 9422 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.755196 9422 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
W20240502 01:59:32.755259 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.755781 9422 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.756016 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.756156 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.756124 9292 raft_consensus.cc:3032] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.756245 9292 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.756261 9020 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.756527 9292 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.756609 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.756686 9292 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.756714 9422 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.756760 9422 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.757414 9292 raft_consensus.cc:2396] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.757511 9422 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.757835 9422 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.758219 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.758337 9157 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.758466 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.758566 9018 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.758601 9018 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.758742 9018 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.758872 9018 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.759276 9157 raft_consensus.cc:2396] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.759449 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.759546 9422 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.759608 9422 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.759647 9018 raft_consensus.cc:2396] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.759922 9360 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.760147 10585 raft_consensus.cc:2726] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.760304 9422 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.760263 8873 catalog_manager.cc:5484] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 2 to 3, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 3 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.760589 9422 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.760886 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.760993 9157 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.761744 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.761915 9157 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 3.
I20240502 01:59:32.761963 9422 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.762056 9422 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.762274 9360 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.762424 10585 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.762552 10585 raft_consensus.cc:686] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
W20240502 01:59:32.762599 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.762687 10585 consensus_queue.cc:237] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.762739 9020 raft_consensus.cc:2396] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.762998 9360 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.763155 9422 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.763301 10585 raft_consensus.cc:2726] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.763610 9422 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.763779 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.763881 9293 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.763921 9293 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.764039 9293 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.764181 9293 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.764506 8871 catalog_manager.cc:5484] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 2 to 3, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 3 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.765007 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.765100 9018 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.765098 9293 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.765578 9359 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.765745 10585 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.765808 10585 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.765911 10585 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.766041 9018 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
W20240502 01:59:32.767541 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.768246 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.768360 9422 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.768416 9422 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.768383 8871 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.769188 9422 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.769505 9422 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.769846 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.769943 9156 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.770620 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.770712 9020 raft_consensus.cc:3032] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.770743 9020 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.770947 9156 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:32.770932 9020 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.9, Last appended by leader: 9, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.771479 9020 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.772593 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.773165 9020 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
W20240502 01:59:32.773202 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.773519 9360 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.774124 11440 raft_consensus.cc:2726] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.774211 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.774430 9292 raft_consensus.cc:3032] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.774482 9292 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.774603 9292 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.774621 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.774717 9422 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.774732 9292 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.774763 9422 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.775444 9422 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.775763 9422 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.775610 9292 raft_consensus.cc:2396] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.776046 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.776137 9293 raft_consensus.cc:3032] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.776175 9293 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.776291 9293 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.776414 9293 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.776788 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.776872 9156 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.777276 9293 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:32.777673 9156 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.777931 9360 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.778482 11440 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.778744 11440 raft_consensus.cc:686] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.778926 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.778972 11440 consensus_queue.cc:237] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.779029 9421 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.779289 9421 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.780114 9421 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
W20240502 01:59:32.781358 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:32.781993 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.782425 9421 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.782614 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.782711 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.782816 9020 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.782860 9020 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.782979 9020 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.783116 9020 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.783130 9157 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.783422 10966 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:32.784039 9020 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 01:59:32.784963 9157 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 01:59:32.785225 9360 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.785617 11441 raft_consensus.cc:2726] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.786523 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.786650 9421 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.786703 9421 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.787560 9421 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.787882 9424 raft_consensus.cc:1226] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20240502 01:59:32.788344 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.788441 9157 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.788718 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.788821 9293 raft_consensus.cc:3032] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.788856 9293 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.789078 9293 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.789230 9293 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.789316 9157 raft_consensus.cc:2396] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 01:59:32.789574 9421 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
W20240502 01:59:32.789206 11430 consensus_queue.cc:1159] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8d72715586b447d69353ef2fdab02ae4" responder_term: 4 status { last_received { term: 3 index: 5 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 3. Current term is 4. Ops: [3.6-3.6]" } } last_received_current_leader { term: 0 index: 0 } }
W20240502 01:59:32.790493 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.791371 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
W20240502 01:59:32.791770 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.791806 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.792011 9293 raft_consensus.cc:2396] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 01:59:32.791471 9421 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.792169 9421 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.792742 9359 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:32.792939 11440 raft_consensus.cc:2726] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.793012 9421 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.793319 9421 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.793751 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.793843 9157 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.794771 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.794857 9157 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 3.
I20240502 01:59:32.794971 9422 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.794988 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.795037 9422 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.795080 9293 raft_consensus.cc:3032] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.795130 9293 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.795140 9360 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.795276 11440 raft_consensus.cc:2781] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.795290 9293 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.795358 11440 raft_consensus.cc:686] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.795416 9293 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.795471 11440 consensus_queue.cc:237] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.795989 9422 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.796233 9293 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 3 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 01:59:32.796262 9422 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.796555 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.796653 9020 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.796905 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.796988 9293 raft_consensus.cc:3032] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.797019 9293 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.797144 9293 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.13, Last appended by leader: 13, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.797252 9293 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.797520 8873 catalog_manager.cc:5484] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.798115 9293 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 3 because replica has last-logged OpId of term: 2 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 12.
I20240502 01:59:32.798504 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.798602 9422 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.798660 9422 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.798736 9020 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 3 because replica has last-logged OpId of term: 1 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 12.
I20240502 01:59:32.798974 9360 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.799355 11441 raft_consensus.cc:2726] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 01:59:32.799897 9422 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.800208 9422 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
W20240502 01:59:32.800201 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.801106 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.801162 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.801282 9153 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.801298 9293 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.801416 9293 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.801599 9293 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.13, Last appended by leader: 13, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.801767 9293 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.802105 9153 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.802579 9293 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.802630 9360 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.802654 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "877896deb7564195b8683801b048c80c"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.802768 9426 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.802834 9426 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.803653 9426 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.803995 9426 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.805035 8871 catalog_manager.cc:5484] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 2 to 3, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 3 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.805182 11441 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
W20240502 01:59:32.805282 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.805366 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.805455 9293 raft_consensus.cc:3032] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.805490 9293 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.805642 9293 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.805763 9293 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.806727 9293 raft_consensus.cc:2396] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.808085 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.808184 9020 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.809046 9020 raft_consensus.cc:2396] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.807878 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.809854 9422 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.809898 9422 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.810910 9422 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.809334 9360 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.811203 9422 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.811575 10585 raft_consensus.cc:2726] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.811657 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.811750 9020 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.812001 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.812542 9020 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:32.812832 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.812930 9422 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.812978 9422 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.813113 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.813259 9293 raft_consensus.cc:3032] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.813335 9293 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
W20240502 01:59:32.813333 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.813536 9293 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.813675 9422 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.813743 9293 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.814091 9422 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.815078 9293 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:32.815064 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.815168 9020 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.815454 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.816217 9359 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.816422 10585 raft_consensus.cc:2726] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.817183 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.817279 9292 raft_consensus.cc:3032] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.817322 9292 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.817466 9292 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.818526 9292 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.819566 9292 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 01:59:32.805260 11441 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.820188 11441 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.13, Last appended by leader: 13, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.821300 9020 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 01:59:32.822062 8871 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.823650 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.823751 9422 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.824599 9360 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
W20240502 01:59:32.824249 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:32.825067 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.823801 9422 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.826093 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.826583 9422 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.827036 9422 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.827320 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.827418 9157 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.828739 9157 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
W20240502 01:59:32.828918 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.828989 9360 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.829315 11441 raft_consensus.cc:2726] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.829401 11441 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.829596 11441 raft_consensus.cc:686] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.829713 11441 consensus_queue.cc:237] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.829913 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.830570 9292 raft_consensus.cc:3032] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.830608 9292 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.830709 9292 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.830824 9292 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.831876 9292 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
W20240502 01:59:32.832549 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43500: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.834239 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.840240 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.840374 9422 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.840451 9422 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.840857 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.841289 9422 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:32.842422 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:32.842748 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.843812 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.844846 9422 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.845379 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.845480 9157 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.846522 9157 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.846765 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.846854 9422 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.846901 9422 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.848927 9422 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.849253 9422 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.849635 9360 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.849884 11441 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.849964 11441 raft_consensus.cc:686] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.850101 11441 consensus_queue.cc:237] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.850481 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.850490 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.850584 9421 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.850586 9022 raft_consensus.cc:3032] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.850587 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.850626 9022 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.850641 9421 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.850759 9022 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 3.16, Last appended by leader: 16, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.850672 9157 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.850908 9022 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.851341 8873 catalog_manager.cc:5484] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:32.851835 9022 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.851897 9157 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.852145 9360 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
W20240502 01:59:32.852952 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.853382 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.854171 9020 raft_consensus.cc:3032] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.854211 9020 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.854502 9020 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.9, Last appended by leader: 9, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.855350 9020 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.857793 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.859210 9020 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 4 because replica has last-logged OpId of term: 3 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
W20240502 01:59:32.859175 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.860337 11440 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.860451 11440 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.860656 11440 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 3.16, Last appended by leader: 16, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
W20240502 01:59:32.861694 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:32.866915 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43500: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.869148 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:32.875025 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:32.876274 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:32.881927 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.889488 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.890913 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:32.892017 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.893869 9421 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.895483 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.895609 9292 raft_consensus.cc:3032] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.895653 9292 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.895794 9292 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.895932 9292 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.897043 9292 raft_consensus.cc:2396] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 8.
I20240502 01:59:32.898587 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.898723 9020 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:32.899051 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.900584 9020 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 3.
I20240502 01:59:32.900868 9360 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.901371 11440 raft_consensus.cc:2781] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.901517 11440 raft_consensus.cc:686] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.901674 11440 consensus_queue.cc:237] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:32.902791 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43500: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.903098 9421 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.898222 8873 catalog_manager.cc:5484] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.903652 8873 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.906836 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.906999 9421 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.907107 9421 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.908282 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.909929 9421 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.910562 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.910733 9020 raft_consensus.cc:3032] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.910822 9020 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.911038 9020 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 3.18, Last appended by leader: 18, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.911259 9020 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.913069 9020 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.913581 9421 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
W20240502 01:59:32.913952 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.913970 9360 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.914141 11441 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.914278 11441 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.914400 11441 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 3.18, Last appended by leader: 18, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.914927 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.915041 9153 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.915208 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.915297 9421 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.915339 9421 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.916018 9153 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.916131 9421 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.916687 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.916817 9292 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.916862 9292 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.916971 9292 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.917131 9292 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:32.917987 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.918099 9292 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
W20240502 01:59:32.918771 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.919194 9359 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:32.919709 11441 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.919893 11441 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.920022 11441 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.921196 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.921401 9020 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.921453 8873 catalog_manager.cc:5484] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 2 to 3, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 3 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.922422 9020 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.922529 9421 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.925374 8873 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.925593 8873 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:32.926244 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.927233 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.927385 9421 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.927510 9421 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 2 FOLLOWER]: Advancing to term 3
W20240502 01:59:32.928401 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.929858 9421 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.933037 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.933145 9292 raft_consensus.cc:3032] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Stepping down as leader of term 2
I20240502 01:59:32.933182 9292 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:32.933296 9292 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.933457 9292 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.933049 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.934322 9153 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 2 FOLLOWER]: Advancing to term 3
I20240502 01:59:32.934422 9292 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 3.
W20240502 01:59:32.934746 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.934921 9359 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
W20240502 01:59:32.935376 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.935710 9153 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 3.
I20240502 01:59:32.935725 11440 raft_consensus.cc:2781] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 01:59:32.935855 11440 raft_consensus.cc:686] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.936017 11440 consensus_queue.cc:237] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:32.936578 9386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.938213 9421 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 3 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.938910 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:32.938867 8873 catalog_manager.cc:5484] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 2 to 3, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 3 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.939008 9421 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.939062 9421 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.939839 9421 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.940156 9421 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:32.940696 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.940804 9020 raft_consensus.cc:3032] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.940846 9020 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:32.940944 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:32.941030 9157 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.940997 9020 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.941200 9020 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.941907 9157 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.942099 9020 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 4.
I20240502 01:59:32.942634 9360 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:32.942880 10585 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.943053 10585 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.943176 10585 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.943971 9020 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 4 index: 19. (index mismatch)
I20240502 01:59:32.943972 9292 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 4 index: 19. (index mismatch)
I20240502 01:59:32.944331 11440 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 01:59:32.944455 10585 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20240502 01:59:32.947485 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43500: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.947894 8871 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 3 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 4 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
W20240502 01:59:32.950942 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.952840 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.953956 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:32.954810 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.955899 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.956053 9157 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.956174 9157 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.956939 9157 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.957057 11097 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:32.957346 9157 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:32.957645 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.957754 9292 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.958051 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.958163 9421 raft_consensus.cc:3032] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:32.958205 9421 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.958077 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.958320 9153 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.958325 9421 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.958460 9153 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.958460 9421 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.958721 9292 raft_consensus.cc:2396] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.959393 9421 raft_consensus.cc:2396] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:32.959666 9089 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.959887 11289 raft_consensus.cc:2726] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:32.959872 9153 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.960206 9153 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.960739 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.960844 9292 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.961087 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.961243 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.961512 9153 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.961367 9020 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.961766 9292 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.961890 9153 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.962095 9089 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.962244 11289 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:32.962317 11289 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.962469 9022 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 3 index: 18. Preceding OpId from leader: term: 4 index: 20. (index mismatch)
I20240502 01:59:32.962427 11289 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.962865 11445 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 01:59:32.963732 9157 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 3 index: 18. Preceding OpId from leader: term: 4 index: 20. (index mismatch)
W20240502 01:59:32.964330 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.965082 8873 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.965437 11441 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 01:59:32.965785 9153 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.966414 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.966516 9022 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.966651 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.966735 9421 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.967535 9421 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.968123 9089 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.968286 11340 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:32.968354 11340 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.968457 11340 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:32.969930 9022 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:32.970283 9020 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:32.970677 9153 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.971241 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.971349 9153 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.971398 9153 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.972175 9153 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.972750 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.972849 9020 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:32.973465 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.973677 9020 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.974180 9090 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
W20240502 01:59:32.974346 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.974346 11289 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:32.974472 11289 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.974578 11289 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:32.974928 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.975878 9153 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.976017 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.976126 9292 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.976420 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.976518 9153 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.976570 9153 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.976970 9292 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.977401 9153 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:32.978120 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.978227 9421 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.978770 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.978863 9022 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.979045 9421 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.979365 9089 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.979506 11340 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:32.979583 11340 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.979822 11340 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:32.979945 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:32.980958 9022 raft_consensus.cc:2396] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:32.981282 9153 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:32.982088 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.982673 8873 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:32.982900 8873 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:32.983090 8873 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.983302 9153 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.983364 9153 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.984181 9153 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.984733 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.984830 9292 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.985704 9292 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 4.
I20240502 01:59:32.985903 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.985996 9022 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:32.985996 9089 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.986161 11340 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:32.986228 11340 raft_consensus.cc:686] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.986330 11340 consensus_queue.cc:237] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.988145 9022 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 8.
I20240502 01:59:32.988507 9153 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.989187 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.989336 9153 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.989435 9153 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:32.989602 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:32.990722 8873 catalog_manager.cc:5484] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 2 to 4, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 4 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:32.991705 9153 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.992029 9153 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:32.992787 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:32.992899 9293 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.992898 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.992985 9153 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.993028 9153 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.993767 9153 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.993870 9293 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:32.994189 9153 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
W20240502 01:59:32.994400 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.994668 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:32.994766 9020 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.995710 9020 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.995812 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.995918 9422 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:32.995923 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43500: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:32.996001 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:32.996246 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:32.996354 9421 raft_consensus.cc:3032] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:32.996389 9421 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:32.996505 9421 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.10, Last appended by leader: 10, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:32.996634 9421 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.996906 9422 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:32.997203 9090 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:32.997355 11289 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:32.997434 11289 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:32.997478 9421 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:32.997553 11289 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:32.998126 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:32.998257 9153 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:32.998308 9153 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:32.998592 9089 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:32.998759 11289 raft_consensus.cc:2726] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 01:59:32.999147 9153 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.000840 9153 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.001103 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.001214 9292 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.001250 9292 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.001324 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.001435 9022 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.001370 9292 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 4.17, Last appended by leader: 17, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.001819 9292 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.002416 9022 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.003086 9090 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.003239 11289 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.003304 11289 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.003242 8873 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.003401 11289 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 3.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.003602 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.003706 9153 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.003756 9153 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.004519 9292 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:33.004527 9153 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.004842 9153 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.005378 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.005416 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.005492 9421 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.005508 9157 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.005555 9157 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.006333 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.006434 9292 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.006441 9157 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.006700 9421 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.006743 9157 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.007040 9089 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.007175 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.007225 11340 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.007272 9422 raft_consensus.cc:3032] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:33.007294 11340 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.007288 9292 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.007309 9422 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.007400 11340 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.007431 9422 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.007550 9422 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.008431 9422 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 01:59:33.008452 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.008543 9292 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.009287 9292 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 01:59:33.009464 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.009541 9089 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.009560 9157 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.009649 9157 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.009701 11289 raft_consensus.cc:2726] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:33.010217 8873 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.010493 9157 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.010768 9157 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.011067 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.011163 9292 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.011348 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.011462 9022 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.011824 9292 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.012040 9089 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.012413 9022 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:33.012642 11289 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.012732 11289 raft_consensus.cc:686] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
W20240502 01:59:33.012745 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.012845 11289 consensus_queue.cc:237] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.013836 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.013926 9157 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.013931 9422 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.013970 9157 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.014339 11340 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.014729 9157 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.015040 9157 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.015280 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.015364 9292 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.015496 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.015591 9421 raft_consensus.cc:3032] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.015626 9421 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.016127 9292 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:33.016472 9421 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.10, Last appended by leader: 10, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.016638 9421 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.018908 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.019016 9157 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.019064 9157 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.019111 9292 raft_consensus.cc:1261] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.019124 9018 raft_consensus.cc:1261] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.019532 11340 consensus_queue.cc:1035] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.019790 11340 consensus_queue.cc:1035] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.019870 9157 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.019878 11340 consensus_queue.cc:1225] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:33.020180 9157 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.020568 9018 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20240502 01:59:33.020625 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.020725 9021 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.021492 9292 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20240502 01:59:33.021528 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.021625 9426 raft_consensus.cc:3032] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.021672 9426 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.021690 9021 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.021795 9426 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.021920 9426 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.022006 9023 pending_rounds.cc:76] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: Aborting all ops after (but not including) 3
I20240502 01:59:33.021966 11288 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 01:59:33.022061 9023 pending_rounds.cc:97] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.022091 9023 raft_consensus.cc:2866] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.022377 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.022477 9157 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.022523 9157 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.022953 9426 raft_consensus.cc:2396] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 01:59:33.023620 9157 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.023895 9157 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.024253 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.024291 11270 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:33.024353 9023 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.024776 9090 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.024993 11288 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.025063 11288 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.025166 11288 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.025295 9023 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.025776 9421 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:33.026326 9090 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.026621 11270 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.026718 11270 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.026786 9089 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.026830 11270 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.026913 11340 raft_consensus.cc:2726] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 01:59:33.027199 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.027302 9293 raft_consensus.cc:3032] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.027340 9293 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.027464 9293 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.027622 9293 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.027729 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.027820 9157 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.027863 9157 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.028542 9293 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.028611 9157 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.028911 9157 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.029101 8873 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.029453 8873 catalog_manager.cc:5484] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.029546 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.029634 9157 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.029690 9157 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.029914 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.030015 9021 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.030750 9157 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.030889 9021 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.031040 9157 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.031620 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.031738 9023 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.032166 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.032655 9023 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.032914 9090 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.033090 11288 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.033156 11288 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.033252 11288 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 3.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.035153 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.035266 9426 raft_consensus.cc:3032] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.035316 9426 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.035451 9426 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.035598 9426 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.035722 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.035867 9157 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.035967 9157 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.036561 9426 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.037204 9020 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.038198 9157 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.038460 9153 raft_consensus.cc:1226] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Rejecting Update request from peer 8d72715586b447d69353ef2fdab02ae4 for earlier term 4. Current term is 5. Ops: []
I20240502 01:59:33.038727 9157 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.039419 9089 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:33.039547 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.039655 9421 raft_consensus.cc:3032] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.039661 11288 raft_consensus.cc:2726] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 01:59:33.039695 9421 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.039757 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 22 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.039832 9422 raft_consensus.cc:3032] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.039889 9422 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.039830 9421 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.17, Last appended by leader: 17, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.039986 9422 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 4.22, Last appended by leader: 22, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.040019 9421 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.040068 9422 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.040117 11262 mvcc.cc:205] Tried to move back new op lower bound from 7023063748649930752 to 7023063748526866432. Current Snapshot: MvccSnapshot[applied={T|T < 7023063748649930752}]
I20240502 01:59:33.040879 9422 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.041184 9089 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.040992 9421 raft_consensus.cc:2396] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:33.041870 11289 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.041945 11289 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.042098 11289 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 4.22, Last appended by leader: 22, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.044395 9293 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 01:59:33.044785 11441 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:33.045248 9020 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
W20240502 01:59:33.046795 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.047171 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 22 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.047279 9022 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.047956 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.048200 11441 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:33.048277 9022 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.048827 11289 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.048928 11289 consensus_queue.cc:1225] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:33.049432 11331 mvcc.cc:205] Tried to move back new op lower bound from 7023063748786573312 to 7023063747649306624. Current Snapshot: MvccSnapshot[applied={T|T < 7023063748786573312 or (T in {7023063748786573312})}]
W20240502 01:59:33.050539 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.051976 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.052038 9021 pending_rounds.cc:76] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: Aborting all ops after (but not including) 3
I20240502 01:59:33.052073 9021 pending_rounds.cc:97] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.052083 9157 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.052098 9021 raft_consensus.cc:2866] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.052141 9157 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.052114 8871 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.052376 8871 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.052589 8871 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.052750 8871 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.053054 9157 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.053594 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.053701 9022 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.054735 9022 raft_consensus.cc:2396] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:33.056699 9153 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20240502 01:59:33.057019 10585 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240502 01:59:33.058197 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.058843 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43500: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.059226 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34090: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.060083 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.060282 9294 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.060431 9157 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.061038 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.061134 9153 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.061179 9153 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.061353 9294 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.062150 9153 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.062476 9153 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
W20240502 01:59:33.062553 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34152: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.062207 9293 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 3 index: 12. (index mismatch)
I20240502 01:59:33.063409 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.063520 9022 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.063658 11441 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 01:59:33.064415 9022 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.064779 9090 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.064922 11340 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.065003 11340 raft_consensus.cc:686] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.065129 11340 consensus_queue.cc:237] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:33.065765 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.066068 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.066172 9089 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:33.066336 11289 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.066408 11289 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.066510 11289 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.066592 9292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.066680 9292 raft_consensus.cc:3032] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.066715 9292 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.066829 9292 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.066955 9292 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.067857 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.068775 9153 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.068850 9153 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.069715 9153 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.069993 9153 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.071717 9292 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.072258 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.072362 9294 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:33.072382 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.072847 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.072913 9292 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20240502 01:59:33.072944 9421 raft_consensus.cc:3032] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:33.072978 9421 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.073107 9421 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.13, Last appended by leader: 13, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.073215 9421 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.073240 9294 raft_consensus.cc:2396] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 3 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 01:59:33.073493 11340 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:33.074142 9421 raft_consensus.cc:2396] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 4 because replica has last-logged OpId of term: 3 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 01:59:33.074218 9422 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20240502 01:59:33.074556 9089 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.074875 11289 raft_consensus.cc:2726] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 01:59:33.074882 11340 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.076630 9293 raft_consensus.cc:1226] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Rejecting Update request from peer 8d72715586b447d69353ef2fdab02ae4 for earlier term 3. Current term is 4. Ops: []
W20240502 01:59:33.077937 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.080053 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.080287 9421 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 22. Preceding OpId from leader: term: 5 index: 24. (index mismatch)
I20240502 01:59:33.080606 11288 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20240502 01:59:33.080660 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.081665 8875 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 3 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.082018 8875 catalog_manager.cc:5484] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.082948 9022 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 22. Preceding OpId from leader: term: 5 index: 24. (index mismatch)
W20240502 01:59:33.083230 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.084014 11340 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20240502 01:59:33.084108 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.084203 9153 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.084262 9153 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.085198 9153 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.085515 9153 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.085827 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.085924 9422 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.085954 9422 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.086097 9422 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 4.14, Last appended by leader: 14, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.086244 9422 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.088151 9422 raft_consensus.cc:2396] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 5 because replica has last-logged OpId of term: 4 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 13.
I20240502 01:59:33.088755 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 5 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.088867 9293 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.089735 9293 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 5.
I20240502 01:59:33.091521 9089 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.091758 11340 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.091835 11340 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.091941 11340 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.13, Last appended by leader: 13, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:33.095017 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.095505 8871 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 4 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 5 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
W20240502 01:59:33.096897 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.098127 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.098244 9023 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.098297 9023 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.099326 9023 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.099673 9023 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.100205 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.100303 9422 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.100431 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.100536 9294 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.100610 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.100713 9023 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.100757 9023 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.101179 9422 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.101456 8956 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.101598 9294 raft_consensus.cc:2396] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:33.101557 9023 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.101634 11264 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.101706 11264 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.101809 11264 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.102219 9023 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.102214 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.102313 9157 raft_consensus.cc:3032] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.102352 9157 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.102464 9157 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.102591 9157 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.102751 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.102836 9291 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.103648 9291 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.103837 8878 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 3 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.104200 8956 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.104457 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.104552 9023 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.104601 9023 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.104665 11264 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.103703 9157 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 01:59:33.104748 11264 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.104861 11264 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.105350 9023 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.105633 9023 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.106002 9293 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 13. Preceding OpId from leader: term: 5 index: 15. (index mismatch)
I20240502 01:59:33.106887 11288 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20240502 01:59:33.107709 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.108603 8878 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:33.108852 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.109520 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.109627 9022 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.109681 9022 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.112931 9022 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:33.113024 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.111645 9422 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 14. Preceding OpId from leader: term: 5 index: 15. (index mismatch)
I20240502 01:59:33.113488 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.113621 9421 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.113690 9421 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.113698 11288 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 01:59:33.113801 11288 consensus_queue.cc:1225] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer 8d72715586b447d69353ef2fdab02ae4 log is divergent from this leader: its last log entry 4.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20240502 01:59:33.113942 9421 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.8, Last appended by leader: 8, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.114285 9421 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.114862 9422 pending_rounds.cc:76] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: Aborting all ops after (but not including) 13
I20240502 01:59:33.115119 9422 pending_rounds.cc:97] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4: Aborting uncommitted NO_OP operation due to leader change: 4.14
I20240502 01:59:33.115240 9422 raft_consensus.cc:2866] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.117223 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.117323 9426 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.118324 9426 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.118638 8956 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.118790 11413 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.118852 11413 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.118969 11413 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:33.119074 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.121542 9421 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.121830 8956 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.121970 11264 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.122082 11264 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.122207 11264 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.123289 9294 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.123356 9020 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.123732 11340 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.124254 11340 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.124385 11340 consensus_queue.cc:1225] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:33.125535 9020 pending_rounds.cc:76] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: Aborting all ops after (but not including) 3
I20240502 01:59:33.125620 9020 pending_rounds.cc:97] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.125684 9020 raft_consensus.cc:2866] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.121909 9022 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.127082 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.127225 9293 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.128373 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34152: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.128976 9291 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 18. Preceding OpId from leader: term: 5 index: 20. (index mismatch)
I20240502 01:59:33.129421 9023 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 18. Preceding OpId from leader: term: 5 index: 20. (index mismatch)
W20240502 01:59:33.129722 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.129700 11340 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 01:59:33.129792 11229 heartbeater.cc:503] Master 127.19.43.254:35279 was elected leader, sending a full tablet report...
I20240502 01:59:33.132227 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.132331 9157 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.132369 9157 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.132483 9157 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.132633 9157 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.133627 9157 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.126873 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.135054 9022 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.135121 9022 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.135222 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.135803 8875 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.136216 9022 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.141676 11340 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 01:59:33.142832 8878 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 4 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.144044 9426 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 4 index: 8. Preceding OpId from leader: term: 5 index: 10. (index mismatch)
I20240502 01:59:33.147822 9293 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.144428 11264 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.146770 11256 mvcc.cc:205] Tried to move back new op lower bound from 7023063749109092352 to 7023063748881723392. Current Snapshot: MvccSnapshot[applied={T|T < 7023063749109092352}]
W20240502 01:59:33.149566 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.152902 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.153013 9157 raft_consensus.cc:3032] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.153053 9157 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.153173 9157 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.153297 9157 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.153417 9291 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 4 index: 8. Preceding OpId from leader: term: 5 index: 10. (index mismatch)
I20240502 01:59:33.154402 9157 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
W20240502 01:59:33.154587 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.155649 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.156069 11352 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240502 01:59:33.160513 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.162007 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.162134 9293 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.163120 9022 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.164265 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.164371 9022 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 01:59:33.170388 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.171375 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.164422 9022 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.179563 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.183876 9293 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.184514 9022 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.184732 8956 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.184948 11352 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.185029 11352 raft_consensus.cc:686] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.185063 9022 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.185166 11352 consensus_queue.cc:237] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.184734 9018 raft_consensus.cc:1226] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Rejecting Update request from peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 for earlier term 5. Current term is 6. Ops: [5.10-5.10]
I20240502 01:59:33.186371 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.186491 9157 raft_consensus.cc:3032] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.186528 9157 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.186655 9157 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.186812 9157 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.187852 9157 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:33.188834 8875 catalog_manager.cc:5484] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 4 to 5, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.191309 11364 mvcc.cc:205] Tried to move back new op lower bound from 7023063749109092352 to 7023063745540849664. Current Snapshot: MvccSnapshot[applied={T|T < 7023063749109092352}]
W20240502 01:59:33.191741 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.193436 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.193542 9018 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.193584 9018 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.194149 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.194260 9293 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.194494 9018 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.194799 9018 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.195298 9293 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:33.195475 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.195569 9294 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.196400 9294 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 5 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 12.
I20240502 01:59:33.196911 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.197016 9426 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.197885 9426 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.198467 8956 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.198676 8956 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.199174 11352 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.199270 11352 raft_consensus.cc:686] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.199396 11352 consensus_queue.cc:237] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.199839 11352 raft_consensus.cc:2726] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.201371 8875 catalog_manager.cc:5484] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 3 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.202483 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.202587 9018 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.202633 9018 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.202801 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.203544 9018 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.203855 9018 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.204157 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.204260 9157 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.204298 9157 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.204416 9157 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.17, Last appended by leader: 17, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.204557 9157 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.205581 9157 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:33.205897 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.206017 9290 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.206727 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.207219 9290 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 4 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
W20240502 01:59:33.208001 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.208835 8956 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
W20240502 01:59:33.209378 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.209431 11264 raft_consensus.cc:2726] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.210875 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.211278 9018 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.211333 9018 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.212307 9018 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.212921 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.213018 9157 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.213055 9157 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.213161 9157 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.213294 9157 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.213343 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.214390 9157 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 01:59:33.215091 9018 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.215507 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.215608 9421 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.216512 9421 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.217181 8956 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
W20240502 01:59:33.217917 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.218245 11413 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.218324 11413 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.218458 11413 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.220252 8875 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
W20240502 01:59:33.223914 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.226572 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.227278 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.228909 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.229020 9020 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.229079 9020 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.230410 9020 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.230968 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 4 candidate_status { last_received { term: 1 index: 13 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.231078 9421 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.231966 9421 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 4.
I20240502 01:59:33.232204 8956 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.232452 11264 raft_consensus.cc:2781] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:33.232533 11264 raft_consensus.cc:686] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 4 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.232643 11264 consensus_queue.cc:237] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.13, Last appended by leader: 13, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.234184 9020 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 4 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.234424 8878 catalog_manager.cc:5484] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 2 to 4, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 4 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.234529 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 4 candidate_status { last_received { term: 1 index: 13 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.234618 9294 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.235653 9294 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 4 because replica has last-logged OpId of term: 2 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 13.
W20240502 01:59:33.237057 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.239145 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.240063 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.240885 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.241001 9020 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.241063 9020 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.242007 9020 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
W20240502 01:59:33.248270 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.254609 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.255563 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.256559 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.262629 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.262769 9422 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.262884 9020 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.263271 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.263379 9157 raft_consensus.cc:3032] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.263411 9157 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.263520 9157 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.10, Last appended by leader: 10, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.263582 9422 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.263638 9157 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.264467 9157 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 01:59:33.264672 8956 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.265002 11413 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.265079 11413 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.265204 11413 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.9, Last appended by leader: 9, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.265221 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.265614 9018 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.265686 9018 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.266841 9018 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.267098 8871 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:33.268652 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.270071 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.270174 9157 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.270231 9157 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.270344 9157 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.267189 9018 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.270488 9157 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.270080 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.271205 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.271304 9018 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.271329 9157 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.271353 9018 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.271566 8957 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.271720 11264 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.271814 11264 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.271922 11264 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.273702 8871 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.277172 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.277290 9294 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.279791 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.280087 9294 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
W20240502 01:59:33.283051 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.283342 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.283613 9018 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.284132 9018 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.284482 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.284602 9157 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.284648 9157 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.284777 9157 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.284947 9157 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.285804 9157 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.286263 8957 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.286557 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.286623 11352 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.286656 9291 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.286774 11352 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.286883 11352 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.287550 9291 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
W20240502 01:59:33.288098 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.288406 8871 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.289213 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.289316 9020 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.289377 9020 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.290263 9020 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.290794 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
W20240502 01:59:33.290822 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.297315 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.297443 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.298269 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.300534 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.303702 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.303815 9422 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.303865 9422 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.303997 9422 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.304163 9422 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.304626 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.305083 9422 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.305322 8956 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.305473 11483 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.308104 11483 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.308291 11483 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
W20240502 01:59:33.309903 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.310092 8878 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 4 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.310863 9020 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.311544 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.311646 9020 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.311671 9018 raft_consensus.cc:1261] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 01:59:33.311729 9020 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.312008 11440 consensus_queue.cc:1035] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.312644 9020 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:33.312850 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.313514 9020 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.313563 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.313673 9294 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.314052 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "877896deb7564195b8683801b048c80c"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.314158 9020 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.314213 9020 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.314635 9294 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 01:59:33.314978 9020 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.315260 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.315371 9157 raft_consensus.cc:3032] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.315408 9157 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.315522 9157 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.315555 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.315644 9157 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.315647 9294 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.316354 9294 raft_consensus.cc:2396] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:33.316499 9290 raft_consensus.cc:1261] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 9. (term mismatch)
I20240502 01:59:33.316551 9290 pending_rounds.cc:76] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: Aborting all ops after (but not including) 8
I20240502 01:59:33.316547 9157 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 01:59:33.316581 9290 pending_rounds.cc:97] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d: Aborting uncommitted NO_OP operation due to leader change: 2.9
I20240502 01:59:33.316632 9290 raft_consensus.cc:2866] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.316994 8957 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.317025 11441 consensus_queue.cc:1035] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.317325 11413 raft_consensus.cc:2726] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.318339 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.318434 9426 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.318531 9020 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
W20240502 01:59:33.318636 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.319348 9426 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.319676 9018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.319770 9018 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.319818 9018 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.320101 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.320712 9018 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.321080 8956 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.321246 11413 raft_consensus.cc:2781] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.321324 11413 raft_consensus.cc:686] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.321436 11413 consensus_queue.cc:237] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.324030 8874 catalog_manager.cc:5484] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 3 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
W20240502 01:59:33.324941 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.325315 9018 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
W20240502 01:59:33.326587 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.327620 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.327725 9290 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.328626 9290 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 5 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.328613 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 5 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.328714 9422 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.329084 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34090: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.329591 9422 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 5.
I20240502 01:59:33.330056 8956 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.330420 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.330755 9022 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.330829 9022 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.331080 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.330474 11413 raft_consensus.cc:2781] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.331458 11413 raft_consensus.cc:686] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
W20240502 01:59:33.331543 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.331590 11413 consensus_queue.cc:237] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.332861 9022 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.333143 8878 catalog_manager.cc:5484] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 3 to 5, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 5 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.333459 9022 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.333478 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.333564 9157 raft_consensus.cc:3032] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.333596 9157 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.333724 9157 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.333829 9157 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.333837 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.333927 9422 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.334056 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.334143 9022 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.334200 9022 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.334614 9157 raft_consensus.cc:2396] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 01:59:33.334761 9422 raft_consensus.cc:2396] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 01:59:33.335785 8956 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.336143 11352 raft_consensus.cc:2726] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.336210 9290 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.336231 9422 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
I20240502 01:59:33.336930 9290 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
I20240502 01:59:33.337234 9022 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.337472 9022 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.337785 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 28 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.337903 9157 raft_consensus.cc:3032] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.337915 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 28 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.337967 9157 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.338019 9290 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.338184 9157 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 5.29, Last appended by leader: 29, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.338393 9157 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.338866 9290 raft_consensus.cc:2396] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 29, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 28.
I20240502 01:59:33.339372 11352 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
W20240502 01:59:33.339475 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.339497 11413 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20240502 01:59:33.340029 9157 raft_consensus.cc:2396] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 29, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 28.
I20240502 01:59:33.340330 8957 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.340559 11413 raft_consensus.cc:2726] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20240502 01:59:33.340873 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.341887 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.341993 9022 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.342074 9022 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.342793 9022 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.343394 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.343497 9018 raft_consensus.cc:1226] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Rejecting Update request from peer efc6b28dd0a34e6f8bb7f4020d2dc6a7 for earlier term 5. Current term is 6. Ops: [5.29-5.29]
I20240502 01:59:33.343533 9157 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.344295 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.344398 9426 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.344916 9157 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.345232 9426 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 01:59:33.345754 8957 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.345913 11352 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.345989 11352 raft_consensus.cc:686] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.346110 11352 consensus_queue.cc:237] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.9, Last appended by leader: 9, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.348021 8873 catalog_manager.cc:5484] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 4 to 6, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.348485 9426 raft_consensus.cc:1261] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.349054 9290 raft_consensus.cc:1261] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.350674 11264 consensus_queue.cc:1035] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.351366 11264 consensus_queue.cc:1035] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.351457 11264 consensus_queue.cc:1225] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer ee7bf4c5836f4264b281adc40ecdac9d log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20240502 01:59:33.352088 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.352288 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.353142 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.353471 9022 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.353628 9290 pending_rounds.cc:76] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: Aborting all ops after (but not including) 3
I20240502 01:59:33.353678 9290 pending_rounds.cc:97] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.353708 9290 raft_consensus.cc:2866] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.357007 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.357282 9022 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.357407 9022 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:33.358224 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.359211 9022 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:33.359894 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.360705 9157 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 10. (index mismatch)
W20240502 01:59:33.363418 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.363274 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.363672 9422 raft_consensus.cc:3032] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 01:59:33.363723 9422 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.364063 9422 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.9, Last appended by leader: 9, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.364241 9422 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 3 FOLLOWER]: Advancing to term 4
W20240502 01:59:33.364503 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.365103 9422 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 4.
I20240502 01:59:33.365600 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 4 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.365711 9290 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 3 FOLLOWER]: Advancing to term 4
I20240502 01:59:33.365605 9294 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 10. (index mismatch)
I20240502 01:59:33.361069 11264 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20240502 01:59:33.366801 9290 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 4.
I20240502 01:59:33.365625 8956 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.370438 11483 raft_consensus.cc:2781] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Leader election won for term 4
I20240502 01:59:33.370519 11483 raft_consensus.cc:686] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 4 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.370632 11483 consensus_queue.cc:237] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.9, Last appended by leader: 9, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.371367 11483 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.359498 9022 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 4 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.373013 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.373171 9022 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 01:59:33.373246 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.373277 9022 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.374173 9022 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.374711 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.374805 9153 raft_consensus.cc:3032] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.374850 9153 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.374963 9153 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.17, Last appended by leader: 17, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.375077 9153 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.375855 9153 raft_consensus.cc:2396] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:33.372500 8873 catalog_manager.cc:5484] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 3 to 4, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 4 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.376608 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.376715 9422 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.376838 9426 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.377364 9294 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.377527 9422 raft_consensus.cc:2396] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 6 because replica has last-logged OpId of term: 4 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:33.377903 11483 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.377990 11483 consensus_queue.cc:1225] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Peer ee7bf4c5836f4264b281adc40ecdac9d log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:33.378067 11264 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.378451 8956 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.378615 11483 raft_consensus.cc:2726] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20240502 01:59:33.380535 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.380609 9294 pending_rounds.cc:76] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: Aborting all ops after (but not including) 3
I20240502 01:59:33.380637 9294 pending_rounds.cc:97] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.380661 9294 raft_consensus.cc:2866] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.381194 9022 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
W20240502 01:59:33.381597 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.382946 9022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.383054 9022 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.383184 9022 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.388078 9022 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.388679 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.388780 9153 raft_consensus.cc:3032] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.388818 9153 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.388851 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.388927 9422 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.388933 9153 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.389073 9153 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.389464 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.389732 9422 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.389859 9153 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 6.
I20240502 01:59:33.390139 8956 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.390282 11264 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.390342 11264 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.390437 11264 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.391997 8875 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 6 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
W20240502 01:59:33.394404 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.395752 9022 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
W20240502 01:59:33.396878 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.399652 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.401187 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.404608 9406 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 01:59:33.407814 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.408560 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.408681 9290 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.408752 9290 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.409780 9290 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.410159 9290 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
W20240502 01:59:33.410722 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.410724 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.410820 9422 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.411665 9422 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.411912 9226 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.412066 11431 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.412070 10764 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:59:33.412174 11431 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.412307 11431 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.417685 8875 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.418118 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.418231 9290 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.418292 9290 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.418838 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.418952 9022 raft_consensus.cc:3032] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.418992 9022 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.419152 9290 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.419147 9022 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.419337 9022 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.419823 9290 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.420006 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.420106 9422 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.420697 9136 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 01:59:33.420953 9422 raft_consensus.cc:2396] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 01:59:33.420972 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.421056 9153 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.422017 9153 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 5.
I20240502 01:59:33.422868 10895 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:59:33.423485 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.423585 9290 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.423616 9227 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4
W20240502 01:59:33.424067 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.424114 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.424314 11435 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.424386 11435 raft_consensus.cc:686] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.424489 11435 consensus_queue.cc:237] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.423635 9290 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
W20240502 01:59:33.425940 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.426191 8875 catalog_manager.cc:5484] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 3 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 5 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.428592 9290 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.429129 9290 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.429175 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.429266 9153 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.430116 9153 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 01:59:33.430405 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.430508 9018 raft_consensus.cc:3032] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.430557 9018 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
W20240502 01:59:33.430625 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.430673 9018 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.430747 11023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:59:33.430801 9018 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.430969 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.431067 9290 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.431114 9290 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.431780 9018 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20240502 01:59:33.436318 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.437613 9227 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.437800 11431 raft_consensus.cc:2726] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 01:59:33.439743 9290 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.441112 9290 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.441725 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.441861 9290 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.441948 9290 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.441913 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.442016 9422 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.442909 9422 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.443203 9226 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.443360 11431 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.443444 11431 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.443557 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.443558 11431 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.443645 9018 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.443681 9018 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.443800 9018 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.443960 9018 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.444823 9018 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.445184 9290 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.446370 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.446480 9153 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.446501 8875 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.447404 9153 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
W20240502 01:59:33.448554 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.449379 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.450155 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.452419 9022 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.452608 9153 raft_consensus.cc:1261] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.453068 9290 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.453850 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.453956 9018 raft_consensus.cc:3032] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.454000 9018 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.454131 9274 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 01:59:33.454146 9018 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.454290 9018 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.453094 11431 consensus_queue.cc:1035] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.455130 9018 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:33.455420 9227 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.456317 11431 raft_consensus.cc:2726] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.457579 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.457690 9290 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.457736 9290 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.458806 9290 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.459344 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.459437 9157 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.460273 9157 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.460538 9227 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.461836 11431 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.461912 11431 raft_consensus.cc:686] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.462018 11431 consensus_queue.cc:237] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.464269 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.464391 9426 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.464561 8873 catalog_manager.cc:5484] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 4 to 6, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.465377 9426 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 10.
I20240502 01:59:33.465710 9290 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.466418 9422 raft_consensus.cc:1261] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 01:59:33.466743 11435 consensus_queue.cc:1035] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.466827 11435 consensus_queue.cc:1225] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Peer 8d72715586b447d69353ef2fdab02ae4 log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:33.467079 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.467209 9290 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.467257 9290 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.467386 9426 pending_rounds.cc:76] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Aborting all ops after (but not including) 3
I20240502 01:59:33.467438 9426 pending_rounds.cc:97] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.467466 9426 raft_consensus.cc:2866] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 01:59:33.467978 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.471451 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.472512 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.474366 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.475101 11163 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:59:33.478287 9022 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20240502 01:59:33.478608 11431 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:33.479143 9001 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 01:59:33.479709 9422 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20240502 01:59:33.480036 11431 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 01:59:33.483618 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.492166 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.492642 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.497293 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.499140 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.502224 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.504448 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.506217 9290 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.507184 9290 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.511099 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 4 index: 17 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.511212 9157 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.512207 9157 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 17.
I20240502 01:59:33.512658 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 4 index: 17 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.512779 9018 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.513779 9018 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
W20240502 01:59:33.514496 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.514794 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.515892 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.514106 9227 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.517911 11435 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.518112 11435 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.518245 11435 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 4.17, Last appended by leader: 17, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.520207 8873 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 7, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 7 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.521445 9294 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.521546 9294 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.521595 9294 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.522594 9294 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.522941 9294 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.523630 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.523730 9022 raft_consensus.cc:3032] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.523761 9022 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.523865 9022 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.10, Last appended by leader: 10, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.523968 9022 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.524869 9022 raft_consensus.cc:2396] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 5 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
W20240502 01:59:33.525544 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.525839 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.525944 9421 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.526847 9294 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.526904 9421 raft_consensus.cc:2454] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 5.
I20240502 01:59:33.526949 9294 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.526993 9294 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.527437 9226 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.527837 9294 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.528157 9294 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.528673 9294 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.528785 9294 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.528837 9294 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
W20240502 01:59:33.529656 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.530359 9294 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.530465 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 15 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.530567 9421 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.530568 11513 raft_consensus.cc:2781] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.530571 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 15 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.530668 9156 raft_consensus.cc:3032] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.530702 9156 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.530813 11513 raft_consensus.cc:686] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.530818 9156 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.15, Last appended by leader: 15, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.530920 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.530987 9156 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.530956 11513 consensus_queue.cc:237] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.9, Last appended by leader: 9, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.531008 9157 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.531358 9421 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.531857 9156 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.531942 9157 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 5.
I20240502 01:59:33.532035 9226 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.532254 11513 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.532858 9227 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.532910 11513 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.533030 11513 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.15, Last appended by leader: 15, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.533157 11435 raft_consensus.cc:2781] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.533233 11435 raft_consensus.cc:686] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.533334 11435 consensus_queue.cc:237] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
W20240502 01:59:33.534307 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.534372 8873 catalog_manager.cc:5484] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 4 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 5 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.535877 9294 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.536415 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.536527 9290 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.536586 9290 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.536878 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 2 index: 12 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.536978 9422 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.537911 9422 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 5 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 12.
W20240502 01:59:33.537954 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.538921 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.539064 9290 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.539786 9290 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.539768 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.539865 9018 raft_consensus.cc:3032] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 4 LEADER]: Stepping down as leader of term 4
I20240502 01:59:33.539896 9018 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.539866 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.539954 9425 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.540006 9018 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.14, Last appended by leader: 14, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.540118 9018 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.540627 9425 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 5.
I20240502 01:59:33.540966 9018 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 5 because replica has last-logged OpId of term: 4 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 13.
I20240502 01:59:33.541003 9226 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.541146 11431 raft_consensus.cc:2781] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.541226 11431 raft_consensus.cc:686] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.541330 11431 consensus_queue.cc:237] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.13, Last appended by leader: 13, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.543434 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.543545 9290 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.543605 9290 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.543406 8873 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.543887 8873 catalog_manager.cc:5484] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 4 to 5, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 5 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.544107 8873 catalog_manager.cc:5484] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 3 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 5 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.545781 9290 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.546286 9290 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.546717 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.546849 9153 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.546892 9153 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.547024 9153 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.547214 9153 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.548166 9153 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.548426 9227 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.548478 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.548571 9421 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.548594 11514 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.550146 9421 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
W20240502 01:59:33.550161 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.550249 11514 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.550386 11514 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.550545 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "877896deb7564195b8683801b048c80c"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
W20240502 01:59:33.551087 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60728: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.551914 8875 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
W20240502 01:59:33.556118 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.558706 9290 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.558784 9290 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.559630 9290 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.560433 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.560534 9018 raft_consensus.cc:3032] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.560596 9018 raft_consensus.cc:729] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
W20240502 01:59:33.561022 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.561390 9018 consensus_queue.cc:260] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.561545 9018 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.562179 9153 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20240502 01:59:33.562862 11264 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 01:59:33.563302 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.567456 9018 raft_consensus.cc:2396] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:33.568539 9294 raft_consensus.cc:1226] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 5. Current term is 6. Ops: [5.10-5.10]
I20240502 01:59:33.569305 9290 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
W20240502 01:59:33.569415 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.569640 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43468: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.569674 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.569764 9426 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.570812 9426 raft_consensus.cc:2396] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:33.570817 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.570904 9290 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.570961 9290 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.571072 9421 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20240502 01:59:33.571247 9226 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d; no voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:33.571525 11483 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 01:59:33.571776 9290 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.572069 9290 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.572463 9426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.572542 9426 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.572654 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.572758 9290 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.572808 9290 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.573374 9426 raft_consensus.cc:2454] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.573650 9226 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.573801 11514 raft_consensus.cc:2726] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.573884 11514 raft_consensus.cc:2781] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.573946 11514 raft_consensus.cc:686] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.574083 11514 consensus_queue.cc:237] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.574090 9290 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:33.574277 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.574467 9290 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.575417 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 5 index: 29 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.575515 9153 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
W20240502 01:59:33.575536 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.576431 9153 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.576711 9227 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.576848 11435 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.576902 11435 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.576990 11435 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 5.29, Last appended by leader: 29, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.577033 8875 catalog_manager.cc:5484] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.577370 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.577462 9018 raft_consensus.cc:3032] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.577498 9018 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.577558 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 5 index: 29 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.577631 9022 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.577607 9018 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.577718 9018 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.578478 9022 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.578591 9018 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 01:59:33.580345 8875 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 7, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 7 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
W20240502 01:59:33.582638 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.583227 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.586455 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.586493 8982 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.587949 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
W20240502 01:59:33.589633 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.590093 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.590188 9290 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.590247 9290 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.590998 9290 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.591315 9290 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.591630 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 6 index: 17 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.591732 9153 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.591745 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 6 index: 17 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.591826 9018 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.591858 9018 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.591997 9018 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 6.17, Last appended by leader: 17, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.592129 9018 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.592568 9153 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.592644 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.592746 9290 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.592795 9290 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.592846 9227 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.592995 11514 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.593097 11514 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.593212 11514 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 6.17, Last appended by leader: 17, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.593287 9018 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.593510 9290 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.594130 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 37 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.594235 9421 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.594451 9290 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.594839 8873 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 6 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 7 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.595027 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.595084 9421 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.595115 9290 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.595162 9290 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.595420 9226 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.595563 11435 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.595639 11435 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.595749 11435 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 5.37, Last appended by leader: 37, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.595949 9290 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.596565 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.596669 9022 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.596727 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.596843 9153 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.597461 9022 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.597697 9153 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 01:59:33.597888 9227 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.597990 9290 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.598143 11435 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.598225 11435 raft_consensus.cc:686] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.598321 11435 consensus_queue.cc:237] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.598376 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 5 index: 37 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.598464 9018 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.598496 9018 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
W20240502 01:59:33.598148 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.599315 9018 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 5.37, Last appended by leader: 37, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.599385 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.599472 9290 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.599498 9018 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.599526 9290 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.599594 8873 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.600155 9290 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:33.600455 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.600746 9421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.600842 9421 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.601586 9421 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 5.
I20240502 01:59:33.601729 9018 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.601819 9226 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.601972 11514 raft_consensus.cc:2781] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Leader election won for term 5
I20240502 01:59:33.602056 11514 raft_consensus.cc:686] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.602162 11514 consensus_queue.cc:237] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.13, Last appended by leader: 13, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.602866 9290 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 5 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.603268 8873 catalog_manager.cc:5484] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 7, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 7 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.603387 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.603487 9290 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.603541 9290 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.603818 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 5 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.603907 9153 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 4 FOLLOWER]: Advancing to term 5
I20240502 01:59:33.604331 9290 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.604619 9153 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 5.
I20240502 01:59:33.604935 9290 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.605340 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.605430 9153 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
W20240502 01:59:33.606135 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.606204 9153 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.606412 9227 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.606550 11435 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.606607 11435 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.606698 11435 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.607822 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.607844 8873 catalog_manager.cc:5484] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 3 to 5, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 5 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.607962 9290 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.608011 9290 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.608093 9153 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 29. Preceding OpId from leader: term: 7 index: 31. (index mismatch)
I20240502 01:59:33.608727 9022 raft_consensus.cc:1261] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 28. Preceding OpId from leader: term: 7 index: 31. (index mismatch)
I20240502 01:59:33.608716 9290 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.608958 11514 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20240502 01:59:33.609028 9290 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.609555 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.609649 9022 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.609648 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.609747 9290 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.609798 9290 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.610116 11514 consensus_queue.cc:1035] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 28, Time since last communication: 0.000s
I20240502 01:59:33.610448 9022 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.610505 9290 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.610713 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.610803 9153 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.610832 9227 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.611038 11514 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.611109 11514 raft_consensus.cc:686] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.611235 11514 consensus_queue.cc:237] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.611506 9153 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 7.
I20240502 01:59:33.611837 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 7 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.611932 9018 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.611968 9018 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.612380 9018 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.612519 9018 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.613309 9018 raft_consensus.cc:2396] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 01:59:33.613308 8875 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 6 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 7 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.613505 8875 catalog_manager.cc:5484] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 7, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 7 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:33.616564 9116 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.617146 9421 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 3 index: 13. Preceding OpId from leader: term: 5 index: 15. (index mismatch)
W20240502 01:59:33.617270 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43478: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.617895 11431 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 01:59:33.618886 9156 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 3 index: 12. Preceding OpId from leader: term: 5 index: 15. (index mismatch)
I20240502 01:59:33.619467 9153 raft_consensus.cc:1261] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 9. Preceding OpId from leader: term: 6 index: 9. (term mismatch)
I20240502 01:59:33.619513 9153 pending_rounds.cc:76] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting all ops after (but not including) 8
I20240502 01:59:33.619534 9153 pending_rounds.cc:97] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7: Aborting uncommitted NO_OP operation due to leader change: 5.9
I20240502 01:59:33.619556 9153 raft_consensus.cc:2866] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.619910 11483 consensus_queue.cc:1035] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.621860 11435 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20240502 01:59:33.622289 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.623512 9421 raft_consensus.cc:1261] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20240502 01:59:33.623908 11483 consensus_queue.cc:1035] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.625536 9290 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.625576 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.625658 9425 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.626492 9425 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 10.
I20240502 01:59:33.626543 9290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
from {username='slave'} at 127.0.0.1:60736
I20240502 01:59:33.626628 9290 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.626674 9290 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.627426 9290 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.627728 9290 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.628017 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.628110 9425 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.628923 9425 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.629900 9226 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.630266 11514 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.630344 11514 raft_consensus.cc:686] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.630447 11514 consensus_queue.cc:237] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:33.631680 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
W20240502 01:59:33.632742 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.632952 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.633054 9157 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.633855 9157 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee7bf4c5836f4264b281adc40ecdac9d in term 6.
I20240502 01:59:33.634686 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.634778 9022 raft_consensus.cc:3032] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.634809 9022 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
W20240502 01:59:33.634843 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.634903 9022 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.634996 9022 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.635782 9022 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ee7bf4c5836f4264b281adc40ecdac9d for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 01:59:33.636575 9290 raft_consensus.cc:1226] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 5. Current term is 6. Ops: []
I20240502 01:59:33.636979 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.637048 9227 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4
I20240502 01:59:33.637084 9425 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.637135 9425 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.637320 8873 catalog_manager.cc:5484] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 5 to 6, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.637970 11514 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.637941 9425 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.638072 11514 raft_consensus.cc:686] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Leader. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.638176 11514 consensus_queue.cc:237] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.638669 9425 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.639048 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 12 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.639554 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.639569 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 12 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.639653 9022 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.640033 9425 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.640098 9425 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
W20240502 01:59:33.640450 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34108: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.641785 9022 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.642089 9360 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.642246 11441 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.642314 11441 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.642408 9018 raft_consensus.cc:1261] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 9. Preceding OpId from leader: term: 7 index: 12. (index mismatch)
I20240502 01:59:33.642426 11441 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.12, Last appended by leader: 12, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.642505 9425 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.642765 9425 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.643010 9290 raft_consensus.cc:3032] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.643098 9290 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.643282 9290 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.12, Last appended by leader: 12, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.643489 9290 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.643919 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.644026 9156 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.644408 9290 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.644882 9156 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.645117 9360 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.645380 10585 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.645455 10585 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.645562 10585 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.645936 9157 raft_consensus.cc:1261] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 10. Preceding OpId from leader: term: 7 index: 12. (index mismatch)
I20240502 01:59:33.646147 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
W20240502 01:59:33.646171 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.646245 9018 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.646281 9018 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.646380 9018 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.646540 9018 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.647344 9018 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.647562 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.647653 9425 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.647711 9425 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.647967 8872 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.648483 9425 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.649235 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.649348 11513 consensus_queue.cc:1035] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 01:59:33.649500 9290 raft_consensus.cc:3032] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.649590 9290 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.649967 9290 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.650599 9426 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 18. (index mismatch)
I20240502 01:59:33.650987 11513 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 01:59:33.651121 9290 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.652006 9290 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 6.
I20240502 01:59:33.652818 9359 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.652969 11441 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.653041 11441 raft_consensus.cc:686] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.653139 11441 consensus_queue.cc:237] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.654316 9153 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 18. (index mismatch)
W20240502 01:59:33.654776 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.654731 8872 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.655068 11513 consensus_queue.cc:1035] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 01:59:33.655625 11513 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 01:59:33.657469 11271 mvcc.cc:205] Tried to move back new op lower bound from 7023063751233961984 to 7023063751112679424. Current Snapshot: MvccSnapshot[applied={T|T < 7023063751233961984}]
W20240502 01:59:33.658231 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.659742 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.659804 9425 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.659847 9157 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
W20240502 01:59:33.659986 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.660375 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.660480 9426 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.660554 9426 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.660717 9157 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 6.
I20240502 01:59:33.661381 9426 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.661731 9426 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.661857 8878 catalog_manager.cc:5484] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 5 to 6, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 6 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.662721 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.662824 9426 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.662886 9426 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.663703 9426 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.664289 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.664393 9158 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.664597 9426 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.665086 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.665182 9421 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.665231 9421 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.665271 9158 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
W20240502 01:59:33.665633 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.666010 9421 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.666357 9421 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.667146 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.667157 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.667241 9421 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.667251 9294 raft_consensus.cc:3032] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.667289 9294 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.667295 9421 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.667387 9294 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.667503 9294 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.668015 9421 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.668303 9421 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.668377 9294 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 01:59:33.669147 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.669262 9158 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.669237 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.669330 9020 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.669376 9020 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.669509 9020 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.669657 9020 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.670044 9158 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.670331 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.670447 9018 raft_consensus.cc:3032] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.670498 9018 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.670555 9020 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.670624 9018 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.10, Last appended by leader: 10, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.670753 9018 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.670928 9360 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.671181 10585 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.671267 10585 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.671379 10585 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.671919 9018 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.672199 9360 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.672405 11441 raft_consensus.cc:2726] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20240502 01:59:33.672724 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.673488 8875 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.673903 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.674008 9421 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.674084 9421 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.675063 9421 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.675608 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 5 index: 15 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.675647 9421 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.675714 9294 raft_consensus.cc:3032] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.675752 9294 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.675866 9294 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 6.16, Last appended by leader: 16, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.675982 9294 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
W20240502 01:59:33.676023 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.676426 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.676512 9426 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.676558 9426 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.676858 9294 raft_consensus.cc:2396] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 15.
I20240502 01:59:33.677368 9426 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.677886 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.677991 9156 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.678261 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 5 index: 15 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.678350 9158 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.678843 9156 raft_consensus.cc:2454] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 6.
I20240502 01:59:33.679057 9158 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.679137 9360 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.679294 11441 raft_consensus.cc:2781] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.679364 11441 raft_consensus.cc:686] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.679464 11441 consensus_queue.cc:237] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.679948 9158 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 38. (index mismatch)
I20240502 01:59:33.680315 11483 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20240502 01:59:33.680771 9360 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.680984 10585 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.681061 10585 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.681164 10585 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.15, Last appended by leader: 15, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.681599 9421 raft_consensus.cc:1261] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 38. (index mismatch)
I20240502 01:59:33.681815 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.681897 9153 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.682214 11483 consensus_queue.cc:1035] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20240502 01:59:33.682675 11262 mvcc.cc:205] Tried to move back new op lower bound from 7023063751233961984 to 7023063751112679424. Current Snapshot: MvccSnapshot[applied={T|T < 7023063751233961984}]
I20240502 01:59:33.684469 8875 catalog_manager.cc:5484] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 5 to 6, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 6 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.684677 8875 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.685211 9153 raft_consensus.cc:2454] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.685478 9360 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.685678 11441 raft_consensus.cc:2781] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.685751 11441 raft_consensus.cc:686] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.685849 11441 consensus_queue.cc:237] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.10, Last appended by leader: 10, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.686931 11331 mvcc.cc:205] Tried to move back new op lower bound from 7023063751380602880 to 7023063750208409600. Current Snapshot: MvccSnapshot[applied={T|T < 7023063751380602880}]
I20240502 01:59:33.689657 9426 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.689946 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.690071 9294 raft_consensus.cc:3032] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.690112 9294 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.690232 9294 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 5.13, Last appended by leader: 13, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.690340 9294 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.690651 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.690752 9421 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.690754 9156 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 10. Preceding OpId from leader: term: 7 index: 12. (index mismatch)
I20240502 01:59:33.690796 9421 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.690972 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.691063 9022 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.691200 9294 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 6 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 01:59:33.691287 9018 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 6 index: 11. Preceding OpId from leader: term: 7 index: 12. (index mismatch)
I20240502 01:59:33.691222 8875 catalog_manager.cc:5484] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.691926 9022 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.692181 9360 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
W20240502 01:59:33.692296 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34158: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.693048 11441 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.693126 11441 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.693244 11441 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 5.13, Last appended by leader: 13, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
W20240502 01:59:33.693603 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.694643 11435 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 01:59:33.694733 11435 consensus_queue.cc:1225] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 6.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20240502 01:59:33.695152 9421 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.695691 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.695786 9294 raft_consensus.cc:3032] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.695791 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.695824 9294 raft_consensus.cc:729] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.695869 9018 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.695930 9294 consensus_queue.cc:260] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.14, Last appended by leader: 14, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.696040 9294 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.696085 11435 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 01:59:33.696727 9018 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 6 because replica has last-logged OpId of term: 4 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 12.
I20240502 01:59:33.696763 9294 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 6 because replica has last-logged OpId of term: 5 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 12.
I20240502 01:59:33.697216 9359 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.697413 11441 raft_consensus.cc:2726] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.697450 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.697530 9290 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.697562 9290 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.697674 9290 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.697811 9290 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.698364 8875 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.698732 9290 raft_consensus.cc:2396] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 01:59:33.698885 9421 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.699249 9022 pending_rounds.cc:76] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: Aborting all ops after (but not including) 10
I20240502 01:59:33.699335 9022 pending_rounds.cc:97] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 6.11
I20240502 01:59:33.699373 9022 raft_consensus.cc:2866] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.699723 9421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.699817 9421 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.700001 9421 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.700732 9421 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.701397 9154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 18 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.701500 9154 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.702172 9154 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.702450 9360 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.702827 11441 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.702953 9421 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.703436 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 18 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.703526 11441 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.703541 9290 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.703641 9290 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.703646 11441 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 6.18, Last appended by leader: 18, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.703918 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "877896deb7564195b8683801b048c80c"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.703949 9290 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 6.18, Last appended by leader: 18, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.704011 9425 raft_consensus.cc:482] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.704062 9425 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.704108 9290 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.704876 9290 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.705111 9425 raft_consensus.cc:504] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.705432 9425 leader_election.cc:288] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.706197 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
W20240502 01:59:33.706478 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.706612 9425 raft_consensus.cc:482] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.706681 9425 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.706786 8875 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.707270 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.707413 9022 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.707521 9425 raft_consensus.cc:504] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.708328 9425 leader_election.cc:288] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.708325 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.708412 9294 raft_consensus.cc:3032] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.708457 9294 raft_consensus.cc:729] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.708570 9294 consensus_queue.cc:260] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.708683 9294 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.708904 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.708994 9425 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.709039 9425 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.709456 9294 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:33.709940 9425 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.710569 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.710673 9294 raft_consensus.cc:3032] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.710711 9294 raft_consensus.cc:729] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.710821 9294 consensus_queue.cc:260] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 6.13, Last appended by leader: 13, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.710934 9294 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.711160 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "877896deb7564195b8683801b048c80c" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.711261 9290 raft_consensus.cc:3037] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.711702 9294 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
W20240502 01:59:33.711766 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43470: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.711987 9290 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.712124 9425 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.712371 9359 leader_election.cc:302] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.712536 11441 raft_consensus.cc:2781] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.712623 11441 raft_consensus.cc:686] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.712731 11441 consensus_queue.cc:237] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.10, Last appended by leader: 10, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.713104 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.713250 9425 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 01:59:33.713114 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56834: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.713349 9425 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.712863 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.714231 9023 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.714320 9425 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.714797 8873 catalog_manager.cc:5484] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d reported cstate change: term changed from 4 to 6, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195). New cstate: current_term: 6 leader_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.714893 9425 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.715001 9023 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 01:59:33.715021 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.715097 9290 raft_consensus.cc:3032] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.715130 9290 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.715466 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.715590 9425 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.715654 9425 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.716707 9360 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
W20240502 01:59:33.716852 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56778: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.709384 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2d727617dd49ad99f95158f6229284" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.717021 9425 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.717291 9018 raft_consensus.cc:3037] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.717402 11441 raft_consensus.cc:2726] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 01:59:33.717674 9425 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.715252 9290 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.717833 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.717944 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.718014 9156 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.718050 9023 raft_consensus.cc:3032] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.718087 9023 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.718012 9290 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.718225 9023 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.10, Last appended by leader: 10, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.718361 9023 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.718798 9156 raft_consensus.cc:2454] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.718897 9290 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 01:59:33.719401 9360 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.719609 11441 raft_consensus.cc:2781] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.719694 11441 raft_consensus.cc:686] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.719797 11441 consensus_queue.cc:237] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.10, Last appended by leader: 10, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.720346 9290 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 6 index: 18. Preceding OpId from leader: term: 7 index: 20. (index mismatch)
I20240502 01:59:33.719255 8872 catalog_manager.cc:5484] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 5 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.720697 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.720743 9156 raft_consensus.cc:1261] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 6 index: 18. Preceding OpId from leader: term: 7 index: 20. (index mismatch)
I20240502 01:59:33.720782 9425 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.720824 9425 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.720871 10585 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 01:59:33.721068 11441 consensus_queue.cc:1035] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 01:59:33.721709 9425 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.722159 9425 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.723143 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 17 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.723250 9157 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.723371 9425 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.723500 9425 raft_consensus.cc:482] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.723588 9425 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.724117 9157 raft_consensus.cc:2396] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 17.
I20240502 01:59:33.724351 9426 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 37. Preceding OpId from leader: term: 6 index: 39. (index mismatch)
I20240502 01:59:33.724390 9425 raft_consensus.cc:504] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.724807 9425 leader_election.cc:288] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.724788 11435 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20240502 01:59:33.725440 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.725472 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.725566 9294 raft_consensus.cc:3032] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.725602 9294 raft_consensus.cc:729] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.725585 9426 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.725642 9426 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.725701 9294 consensus_queue.cc:260] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.10, Last appended by leader: 10, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.725817 9294 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.726241 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 17 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.726341 9021 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.726449 9426 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.726712 9294 raft_consensus.cc:2396] T 3c5477d6583246d492f1171f96cf6260 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 01:59:33.726737 9426 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.727243 9021 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.727865 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5477d6583246d492f1171f96cf6260" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 3 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.727912 9360 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.727957 9021 raft_consensus.cc:3037] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.728207 9157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 38 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.727962 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 38 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.728293 9157 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.728305 9020 raft_consensus.cc:3032] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.728338 9020 raft_consensus.cc:729] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:33.728422 11440 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.728441 9020 consensus_queue.cc:260] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 6.39, Last appended by leader: 39, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.728487 11440 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.728562 9020 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.728576 11440 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 4.17, Last appended by leader: 17, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.728760 9021 raft_consensus.cc:2396] T 3c5477d6583246d492f1171f96cf6260 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 6 because replica has last-logged OpId of term: 4 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 9.
I20240502 01:59:33.729097 9157 raft_consensus.cc:2396] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 39, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 38.
I20240502 01:59:33.729411 9020 raft_consensus.cc:2396] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 39, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 38.
I20240502 01:59:33.729617 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.726330 9023 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
W20240502 01:59:33.730240 9250 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.730815 9360 leader_election.cc:302] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.731009 9360 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:33.731104 9018 raft_consensus.cc:2396] T ce2d727617dd49ad99f95158f6229284 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 01:59:33.731276 9360 leader_election.cc:302] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4; no voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.731346 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.731491 9153 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.731600 11441 raft_consensus.cc:2726] T 3c5477d6583246d492f1171f96cf6260 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 01:59:33.731645 10585 raft_consensus.cc:2726] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 01:59:33.731676 11441 raft_consensus.cc:2726] T ce2d727617dd49ad99f95158f6229284 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20240502 01:59:33.732174 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.733001 9018 raft_consensus.cc:1261] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 5 index: 37. Preceding OpId from leader: term: 6 index: 39. (index mismatch)
I20240502 01:59:33.733131 9426 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.733193 9426 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.733956 9426 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.734705 9426 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.734712 8872 catalog_manager.cc:5484] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.735074 11435 consensus_queue.cc:1035] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20240502 01:59:33.735219 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 39 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.735227 9426 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.735301 9023 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.735316 9426 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.735364 9426 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.736336 9023 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.736276 9426 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.736855 9426 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.736958 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.737044 9294 raft_consensus.cc:3032] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Stepping down as leader of term 5
I20240502 01:59:33.737079 9294 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.737123 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 6 index: 39 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.737102 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 6 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.737201 9156 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.737201 9290 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.737324 9294 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.737478 9294 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 5 FOLLOWER]: Advancing to term 6
I20240502 01:59:33.737236 9290 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.737519 9422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
from {username='slave'} at 127.0.0.1:34136
I20240502 01:59:33.737609 9422 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.737653 9422 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.737648 9290 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.39, Last appended by leader: 39, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.737752 9290 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.738431 9421 raft_consensus.cc:1226] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Rejecting Update request from peer d6bd594b464a44f89a636ca1b758aefb for earlier term 6. Current term is 7. Ops: [6.39-6.39]
I20240502 01:59:33.738517 9290 raft_consensus.cc:2454] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.738492 9422 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
W20240502 01:59:33.738561 11364 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.105s ago.
I20240502 01:59:33.738768 9359 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.738782 9422 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:33.739004 11441 raft_consensus.cc:2781] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.739085 11441 raft_consensus.cc:686] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.739085 9294 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 6.
I20240502 01:59:33.739095 9153 raft_consensus.cc:2454] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.739202 11441 consensus_queue.cc:237] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.39, Last appended by leader: 39, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.739368 9359 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:33.739629 11527 raft_consensus.cc:2781] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 01:59:33.739738 9156 raft_consensus.cc:2454] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 6.
I20240502 01:59:33.739810 9022 raft_consensus.cc:2454] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.739845 11527 raft_consensus.cc:686] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.739997 11527 consensus_queue.cc:237] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.740510 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.740612 9360 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:33.740667 9022 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.740804 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "8d72715586b447d69353ef2fdab02ae4" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:33.741170 11527 raft_consensus.cc:2781] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.741377 9156 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.741540 9022 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d72715586b447d69353ef2fdab02ae4 in term 7.
I20240502 01:59:33.742280 9156 raft_consensus.cc:2396] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d72715586b447d69353ef2fdab02ae4 for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 01:59:33.742290 9360 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:33.742713 11441 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.742810 11441 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.742919 11441 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.742898 8872 catalog_manager.cc:5484] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.742960 11527 raft_consensus.cc:686] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Leader. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.743080 8872 catalog_manager.cc:5484] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 5 to 6, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 6 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.743177 9290 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 18. (index mismatch)
I20240502 01:59:33.743218 11527 consensus_queue.cc:237] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.743491 11440 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 01:59:33.744302 9156 raft_consensus.cc:1261] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 6 index: 18. (index mismatch)
I20240502 01:59:33.744639 9022 raft_consensus.cc:1226] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 6. Current term is 7. Ops: [6.38-6.39]
I20240502 01:59:33.744617 11440 consensus_queue.cc:1035] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 01:59:33.745404 8872 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 5 to 7, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
W20240502 01:59:33.751631 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:33.751883 9294 raft_consensus.cc:1261] T 877896deb7564195b8683801b048c80c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 5 index: 9. Preceding OpId from leader: term: 7 index: 12. (index mismatch)
I20240502 01:59:33.751931 9022 raft_consensus.cc:1261] T 877896deb7564195b8683801b048c80c P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 5 index: 10. Preceding OpId from leader: term: 7 index: 12. (index mismatch)
I20240502 01:59:33.752305 11441 consensus_queue.cc:1035] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.752470 11440 consensus_queue.cc:1035] T 877896deb7564195b8683801b048c80c P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 01:59:33.752720 8875 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 5 to 7, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.752920 8875 catalog_manager.cc:5484] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 reported cstate change: term changed from 6 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196). New cstate: current_term: 7 leader_uuid: "8d72715586b447d69353ef2fdab02ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.758976 9022 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 6 index: 12. Preceding OpId from leader: term: 7 index: 14. (index mismatch)
I20240502 01:59:33.759387 9291 raft_consensus.cc:1261] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 6 index: 12. Preceding OpId from leader: term: 7 index: 14. (index mismatch)
I20240502 01:59:33.759580 11441 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 01:59:33.760371 10585 consensus_queue.cc:1035] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 01:59:33.766247 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9b28feae4b68470097028024d908ed23"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.766358 9153 raft_consensus.cc:482] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.766422 9153 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.767315 9153 raft_consensus.cc:504] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.767648 9153 leader_election.cc:288] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.767961 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.768110 9425 raft_consensus.cc:3032] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.768198 9425 raft_consensus.cc:729] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.768234 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b28feae4b68470097028024d908ed23" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.768344 9294 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.769328 9294 raft_consensus.cc:2454] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 7.
I20240502 01:59:33.769644 9089 leader_election.cc:302] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.769917 11289 raft_consensus.cc:2781] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.769995 11289 raft_consensus.cc:686] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.770197 11289 consensus_queue.cc:237] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.771553 8875 catalog_manager.cc:5484] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 6 to 7, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 7 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
W20240502 01:59:33.777196 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.778698 9425 consensus_queue.cc:260] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.778894 9425 raft_consensus.cc:3037] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.781304 9023 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 6 index: 17. Preceding OpId from leader: term: 7 index: 19. (index mismatch)
I20240502 01:59:33.783063 9156 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Refusing update from remote peer ee7bf4c5836f4264b281adc40ecdac9d: Log matching property violated. Preceding OpId in replica: term: 6 index: 17. Preceding OpId from leader: term: 7 index: 19. (index mismatch)
I20240502 01:59:33.783926 11513 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 01:59:33.784370 9425 raft_consensus.cc:2396] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 01:59:33.785018 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.785885 9153 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.785884 11512 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 01:59:33.786476 9153 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.788479 9153 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.789011 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.789119 9290 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.790572 9290 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.790817 9089 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.790980 11289 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.791056 11289 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.791193 9153 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.791198 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.791168 11289 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.791288 9018 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.792228 9018 raft_consensus.cc:2396] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20240502 01:59:33.792855 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.792979 9157 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.793042 9157 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.793953 9157 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.794292 9157 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
W20240502 01:59:33.794598 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.795328 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.795423 9020 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.796322 9020 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 01:59:33.793203 8872 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 6 to 8, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.797508 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.797715 9157 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.797773 9157 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.798710 9157 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.799260 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.799350 9018 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.800110 9018 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.801713 9090 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.801877 11270 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.801952 11270 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.802090 11270 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 7.14, Last appended by leader: 14, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.802857 9157 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.803428 9157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.803550 9157 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.804653 9157 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.804774 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.804886 9291 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.804936 9291 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.805063 9291 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 7.14, Last appended by leader: 14, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.805173 9291 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.805704 9023 raft_consensus.cc:1226] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 7. Current term is 8. Ops: []
I20240502 01:59:33.805858 9157 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.806049 9293 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 6 index: 16. Preceding OpId from leader: term: 7 index: 17. (index mismatch)
I20240502 01:59:33.806077 9291 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.806577 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.806572 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.806677 9020 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.806679 9422 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.806731 9422 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.806850 9422 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.806977 9422 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.807480 11441 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 01:59:33.807543 9020 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.807574 11441 consensus_queue.cc:1225] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Peer ee7bf4c5836f4264b281adc40ecdac9d log is divergent from this leader: its last log entry 6.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20240502 01:59:33.807955 9422 raft_consensus.cc:2396] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20240502 01:59:33.808095 9294 pending_rounds.cc:76] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: Aborting all ops after (but not including) 15
I20240502 01:59:33.808130 9294 pending_rounds.cc:97] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d: Aborting uncommitted NO_OP operation due to leader change: 6.16
I20240502 01:59:33.808115 9090 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.808156 9294 raft_consensus.cc:2866] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.808261 11289 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.808329 11289 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.808435 11289 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.809157 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.809257 9425 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.810127 9425 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.810371 9089 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:33.810698 11270 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.810597 8872 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.810777 11270 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.810875 11270 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.10, Last appended by leader: 10, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.811380 9157 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.812230 9156 raft_consensus.cc:1261] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 5 index: 15. Preceding OpId from leader: term: 7 index: 17. (index mismatch)
W20240502 01:59:33.812451 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60746: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.813211 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.813308 9158 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.813356 9158 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.814203 9158 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.814483 9158 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.814594 8872 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.814792 8872 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 6 to 8, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.815701 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.815704 10585 consensus_queue.cc:1035] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 01:59:33.815796 9293 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.816519 9293 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 7.
I20240502 01:59:33.816612 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.816717 9153 raft_consensus.cc:482] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.816766 9153 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.816799 9089 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.816951 11289 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:33.817019 11289 raft_consensus.cc:686] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.817121 11289 consensus_queue.cc:237] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.818543 9153 raft_consensus.cc:504] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
W20240502 01:59:33.818899 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56812: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.819700 9153 leader_election.cc:288] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.820212 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.820307 9293 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.821094 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.821200 9153 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.821214 9293 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:33.821262 9153 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.821657 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.821794 9023 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.822156 9153 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
W20240502 01:59:33.822269 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60762: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.822688 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.822782 9020 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.822814 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eacd063e75d4558998915f5ba599fba" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.822911 9425 raft_consensus.cc:3032] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.822947 9425 raft_consensus.cc:729] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.823081 9023 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 01:59:33.823060 9425 consensus_queue.cc:260] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.11, Last appended by leader: 11, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.823176 9425 raft_consensus.cc:3037] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.823503 9020 raft_consensus.cc:2454] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.823931 9090 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.824092 11289 raft_consensus.cc:2781] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.824136 9425 raft_consensus.cc:2396] T 9eacd063e75d4558998915f5ba599fba P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:33.824173 11289 raft_consensus.cc:686] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.824286 11289 consensus_queue.cc:237] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.824440 9089 leader_election.cc:302] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.824604 9156 raft_consensus.cc:1226] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 7. Current term is 8. Ops: []
I20240502 01:59:33.824779 11289 raft_consensus.cc:2726] T 9eacd063e75d4558998915f5ba599fba P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 01:59:33.825332 8872 catalog_manager.cc:5484] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 5 to 7, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 7 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
W20240502 01:59:33.825754 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56798: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.826292 9020 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 5 index: 13. Preceding OpId from leader: term: 7 index: 15. (index mismatch)
I20240502 01:59:33.826622 11528 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 01:59:33.826728 9293 raft_consensus.cc:1261] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Refusing update from remote peer 8d72715586b447d69353ef2fdab02ae4: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 15. (index mismatch)
I20240502 01:59:33.827251 11528 consensus_queue.cc:1035] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 01:59:33.827355 11528 consensus_queue.cc:1225] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Peer ee7bf4c5836f4264b281adc40ecdac9d log is divergent from this leader: its last log entry 6.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20240502 01:59:33.827320 9153 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.827816 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.827926 9293 pending_rounds.cc:76] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: Aborting all ops after (but not including) 13
I20240502 01:59:33.827980 9293 pending_rounds.cc:97] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d: Aborting uncommitted NO_OP operation due to leader change: 6.14
I20240502 01:59:33.828018 9293 raft_consensus.cc:2866] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.827965 9425 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.828166 9425 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.828440 9425 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.828632 9425 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
W20240502 01:59:33.830477 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.830627 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.830750 9153 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.830807 9153 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.831300 9425 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20240502 01:59:33.831713 9153 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.832257 9153 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.832440 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.832531 9020 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.832512 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.832603 9294 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.832641 9294 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.832767 9294 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 7.18, Last appended by leader: 18, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.832891 9294 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.832891 9156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.832974 9156 raft_consensus.cc:482] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.833019 9156 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.833356 9020 raft_consensus.cc:2454] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.833668 9090 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.833781 9294 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 17.
I20240502 01:59:33.833755 9156 raft_consensus.cc:504] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.834545 9156 leader_election.cc:288] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.834829 11270 raft_consensus.cc:2781] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.834957 11270 raft_consensus.cc:686] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.835067 11270 consensus_queue.cc:237] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.835129 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 17 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.835216 9425 raft_consensus.cc:3032] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.835253 9425 raft_consensus.cc:729] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.835359 9425 consensus_queue.cc:260] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 7.17, Last appended by leader: 17, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.835490 9425 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.835513 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf038eed2c6645d8a7dbf126e4013b97" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 17 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.836184 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.836282 9158 raft_consensus.cc:482] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.836330 9158 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.836333 9425 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.835592 9293 raft_consensus.cc:3037] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.837134 9158 raft_consensus.cc:504] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.837396 9158 leader_election.cc:288] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.838670 9293 raft_consensus.cc:2454] T cf038eed2c6645d8a7dbf126e4013b97 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.838886 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.838985 9158 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.839033 9158 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.839313 9020 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 14. Preceding OpId from leader: term: 8 index: 16. (index mismatch)
I20240502 01:59:33.839650 11289 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 01:59:33.841024 9158 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.841498 9158 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.841635 9089 leader_election.cc:302] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.841755 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.841845 9290 raft_consensus.cc:3032] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.841871 11289 raft_consensus.cc:2781] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.841887 9290 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.841953 11289 raft_consensus.cc:686] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.842015 9290 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.21, Last appended by leader: 21, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.842146 9290 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.842100 11289 consensus_queue.cc:237] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 7.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.842906 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.842969 9290 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 20.
I20240502 01:59:33.842993 9291 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.843300 9294 raft_consensus.cc:1261] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 14. Preceding OpId from leader: term: 8 index: 16. (index mismatch)
I20240502 01:59:33.843812 9291 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 7 because replica has last-logged OpId of term: 5 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 01:59:33.844611 11270 consensus_queue.cc:1035] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:33.845561 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "542bd878941942959df6a4c652f19c95"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.845669 9158 raft_consensus.cc:482] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.845721 9158 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.846210 8875 catalog_manager.cc:5484] T cf038eed2c6645d8a7dbf126e4013b97 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.846316 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.846407 9023 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.846434 8875 catalog_manager.cc:5484] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:33.846542 9158 raft_consensus.cc:504] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.846638 8875 catalog_manager.cc:5484] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.846918 9158 leader_election.cc:288] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.847388 9023 raft_consensus.cc:2454] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.847487 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.847575 9422 raft_consensus.cc:3032] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.847558 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542bd878941942959df6a4c652f19c95" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.847612 9422 raft_consensus.cc:729] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.847635 9090 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.847645 9291 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.847725 9422 consensus_queue.cc:260] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.11, Last appended by leader: 11, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.847812 11289 raft_consensus.cc:2781] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.847868 9422 raft_consensus.cc:3037] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.847874 11289 raft_consensus.cc:686] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.847970 11289 consensus_queue.cc:237] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.20, Last appended by leader: 20, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.848464 9291 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:33.848747 9422 raft_consensus.cc:2396] T 542bd878941942959df6a4c652f19c95 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 9.
I20240502 01:59:33.849249 9089 leader_election.cc:302] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.849470 11270 raft_consensus.cc:2726] T 542bd878941942959df6a4c652f19c95 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 01:59:33.849826 9023 raft_consensus.cc:1226] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 7. Current term is 8. Ops: [7.21-7.21]
I20240502 01:59:33.850514 9158 raft_consensus.cc:1226] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Rejecting Update request from peer ee7bf4c5836f4264b281adc40ecdac9d for earlier term 7. Current term is 8. Ops: [7.21-7.21]
I20240502 01:59:33.854717 8875 catalog_manager.cc:5484] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.855695 9023 raft_consensus.cc:1261] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20240502 01:59:33.855985 11540 consensus_queue.cc:1035] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.856074 11540 consensus_queue.cc:1225] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 01:59:33.856773 9023 pending_rounds.cc:76] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Aborting all ops after (but not including) 3
I20240502 01:59:33.856828 9023 pending_rounds.cc:97] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 01:59:33.856858 9023 raft_consensus.cc:2866] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.856920 9023 pending_rounds.cc:97] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20240502 01:59:33.857012 9023 raft_consensus.cc:2866] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.859918 9294 raft_consensus.cc:1261] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20240502 01:59:33.860999 11289 consensus_queue.cc:1035] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 01:59:33.863346 11256 mvcc.cc:205] Tried to move back new op lower bound from 7023063752107225088 to 7023063748455452672. Current Snapshot: MvccSnapshot[applied={T|T < 7023063752107225088}]
W20240502 01:59:33.873172 9385 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
W20240502 01:59:33.879788 8983 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56822: Illegal state: replica d6bd594b464a44f89a636ca1b758aefb is not leader of this config: current role FOLLOWER
I20240502 01:59:33.880575 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb2480d58284153a9ea5d7dc3b9960c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.880745 9425 raft_consensus.cc:3032] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.880792 9425 raft_consensus.cc:729] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.880939 9425 consensus_queue.cc:260] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 6.13, Last appended by leader: 13, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.881150 9425 raft_consensus.cc:3037] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.886240 9425 raft_consensus.cc:2396] T ecb2480d58284153a9ea5d7dc3b9960c P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 7 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 11.
I20240502 01:59:33.886521 9089 leader_election.cc:302] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.886756 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "462a0918c439471791f181bf6022d6a4"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.886865 9153 raft_consensus.cc:482] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.886927 9153 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.888408 9018 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20240502 01:59:33.888743 11289 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:33.889097 9153 raft_consensus.cc:504] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.889711 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.889829 9020 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.890738 9020 raft_consensus.cc:2454] T 462a0918c439471791f181bf6022d6a4 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.891728 9090 leader_election.cc:302] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.891844 9422 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20240502 01:59:33.891974 11270 raft_consensus.cc:2781] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.892041 11270 raft_consensus.cc:686] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.892150 11270 consensus_queue.cc:237] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.893138 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462a0918c439471791f181bf6022d6a4" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.893244 9422 raft_consensus.cc:3032] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.893281 9422 raft_consensus.cc:729] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.893384 9422 consensus_queue.cc:260] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.893503 9422 raft_consensus.cc:3037] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.894551 9422 raft_consensus.cc:2396] T 462a0918c439471791f181bf6022d6a4 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:33.895090 11540 raft_consensus.cc:2726] T ecb2480d58284153a9ea5d7dc3b9960c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 01:59:33.895084 8875 catalog_manager.cc:5484] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.897302 11270 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 01:59:33.897398 11270 consensus_queue.cc:1225] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer 8d72715586b447d69353ef2fdab02ae4 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20240502 01:59:33.899611 9422 pending_rounds.cc:76] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: Aborting all ops after (but not including) 7
I20240502 01:59:33.899667 9422 pending_rounds.cc:97] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20240502 01:59:33.899708 9422 raft_consensus.cc:2866] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.899889 9153 leader_election.cc:288] T 462a0918c439471791f181bf6022d6a4 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.901027 9422 raft_consensus.cc:1261] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20240502 01:59:33.902563 9294 raft_consensus.cc:1261] T 9b28feae4b68470097028024d908ed23 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20240502 01:59:33.903080 11270 consensus_queue.cc:1035] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.903180 11270 consensus_queue.cc:1225] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Peer 8d72715586b447d69353ef2fdab02ae4 log is divergent from this leader: its last log entry 6.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20240502 01:59:33.903821 9422 pending_rounds.cc:76] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Aborting all ops after (but not including) 8
I20240502 01:59:33.903870 9422 pending_rounds.cc:97] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20240502 01:59:33.903900 9422 raft_consensus.cc:2866] T 9b28feae4b68470097028024d908ed23 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 01:59:33.904167 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.904273 9153 raft_consensus.cc:482] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.904325 9153 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.904484 11270 consensus_queue.cc:1035] T 9b28feae4b68470097028024d908ed23 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 01:59:33.905261 9153 raft_consensus.cc:504] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.905876 9153 leader_election.cc:288] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.906780 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 37 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.906894 9020 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.907306 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e3336bcbb54cb0a15f00209445713d" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 37 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.907407 9294 raft_consensus.cc:3032] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.907439 9294 raft_consensus.cc:729] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.907552 9294 consensus_queue.cc:260] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 7.37, Last appended by leader: 37, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.907699 9294 raft_consensus.cc:3037] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.908048 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.908142 9153 raft_consensus.cc:482] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.908190 9153 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.908707 9294 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.909242 9153 raft_consensus.cc:504] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.910174 9089 leader_election.cc:302] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.910364 11270 raft_consensus.cc:2781] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.910439 11270 raft_consensus.cc:686] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.910537 11270 consensus_queue.cc:237] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 7.37, Last appended by leader: 37, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.910737 9020 raft_consensus.cc:2454] T d1e3336bcbb54cb0a15f00209445713d P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.912679 9153 leader_election.cc:288] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545)
I20240502 01:59:33.913518 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.913617 9153 raft_consensus.cc:482] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.913677 9153 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.913724 9020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.913815 9020 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.913867 8875 catalog_manager.cc:5484] T d1e3336bcbb54cb0a15f00209445713d P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.914299 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9928d7f049d648ea84508474dd42cdc2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.914407 9422 raft_consensus.cc:3032] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.914455 9422 raft_consensus.cc:729] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.914477 9153 raft_consensus.cc:504] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.914588 9422 consensus_queue.cc:260] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 7.18, Last appended by leader: 18, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.914670 9020 raft_consensus.cc:2454] T 9928d7f049d648ea84508474dd42cdc2 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.914718 9422 raft_consensus.cc:3037] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.914937 9090 leader_election.cc:302] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.915148 11270 raft_consensus.cc:2781] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.915212 11270 raft_consensus.cc:686] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.915315 11270 consensus_queue.cc:237] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 5.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:33.915714 9422 raft_consensus.cc:2396] T 9928d7f049d648ea84508474dd42cdc2 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 17.
I20240502 01:59:33.915519 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.916263 9153 leader_election.cc:288] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.917291 9018 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.918246 9018 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.921133 8875 catalog_manager.cc:5484] T 9928d7f049d648ea84508474dd42cdc2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:33.923394 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec2c7377d096437eb6e530e2086a67c7" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.923493 9422 raft_consensus.cc:3032] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.923534 9422 raft_consensus.cc:729] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.923635 9422 consensus_queue.cc:260] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.924000 9422 raft_consensus.cc:3037] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.925917 9422 raft_consensus.cc:2396] T ec2c7377d096437eb6e530e2086a67c7 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 01:59:33.927630 9153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c22a74b9931b46acb4077befa2771906"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.927855 9153 raft_consensus.cc:482] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.930330 9153 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.931512 9153 raft_consensus.cc:504] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.932050 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 39 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.932317 9018 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.932837 9089 leader_election.cc:302] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb
I20240502 01:59:33.933173 11289 raft_consensus.cc:2726] T ec2c7377d096437eb6e530e2086a67c7 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 01:59:33.940004 9018 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.940829 9153 leader_election.cc:288] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.941915 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c22a74b9931b46acb4077befa2771906" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 6 index: 39 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.942013 9422 raft_consensus.cc:3037] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.942306 9090 leader_election.cc:302] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.942718 9156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.942816 9156 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.942865 9156 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.942857 9422 raft_consensus.cc:2454] T c22a74b9931b46acb4077befa2771906 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.943639 9156 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.943912 9156 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.944321 11289 raft_consensus.cc:2781] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.944392 11289 raft_consensus.cc:686] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.944485 11289 consensus_queue.cc:237] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 6.39, Last appended by leader: 39, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.945415 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.945613 9158 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.946238 8872 catalog_manager.cc:5484] T c22a74b9931b46acb4077befa2771906 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 6 to 8, leader changed from d6bd594b464a44f89a636ca1b758aefb (127.19.43.193) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.946906 9022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 23 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.947011 9022 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.947567 9158 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.947867 9022 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.950248 9158 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.950791 9158 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers d6bd594b464a44f89a636ca1b758aefb (127.19.43.193:36545), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.951359 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.951503 9158 raft_consensus.cc:482] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.951603 9158 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.952438 9158 raft_consensus.cc:504] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.953851 9158 leader_election.cc:288] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.954293 9090 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.954746 9023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
I20240502 01:59:33.954842 9023 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.954967 11539 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.955044 11539 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.955152 11539 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 7.23, Last appended by leader: 23, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.955639 9023 raft_consensus.cc:2454] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.955958 9090 leader_election.cc:302] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:33.957758 11270 raft_consensus.cc:2781] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.957849 11270 raft_consensus.cc:686] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.957957 11270 consensus_queue.cc:237] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.958325 8872 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:33.959676 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 23 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.959791 9293 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.959849 9293 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.959982 9289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 25 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.959980 9293 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 7.23, Last appended by leader: 23, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.960078 9289 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.960139 9293 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.960952 9293 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.961078 9289 raft_consensus.cc:2396] T 3ec3c859176746d697e47b54b2d5b9a2 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 7 because replica has last-logged OpId of term: 6 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 25.
I20240502 01:59:33.961244 9153 raft_consensus.cc:1226] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Rejecting Update request from peer 8d72715586b447d69353ef2fdab02ae4 for earlier term 6. Current term is 7. Ops: [6.26-6.26]
I20240502 01:59:33.959682 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.961984 9290 raft_consensus.cc:3032] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.962184 9290 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ee7bf4c5836f4264b281adc40ecdac9d, State: Running, Role: LEADER
I20240502 01:59:33.962337 9290 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:33.962466 9290 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.963068 11527 consensus_queue.cc:1046] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: INVALID_TERM, Last received: 6.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20240502 01:59:33.963378 9290 raft_consensus.cc:2396] T 0f04b862773348a78aa4596b0f42d573 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:33.964993 9158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6"
dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
from {username='slave'} at 127.0.0.1:43416
I20240502 01:59:33.965102 9158 raft_consensus.cc:482] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.965162 9158 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.965435 8872 catalog_manager.cc:5484] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:33.963624 11527 raft_consensus.cc:3032] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 01:59:33.965762 11527 raft_consensus.cc:729] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.965879 11527 consensus_queue.cc:260] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 6.26, Last appended by leader: 26, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.966049 11527 raft_consensus.cc:3037] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:33.966369 9158 raft_consensus.cc:504] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.966815 9158 leader_election.cc:288] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:33.967167 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 30 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.967262 9422 raft_consensus.cc:3032] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:33.967303 9422 raft_consensus.cc:729] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:33.967413 9422 consensus_queue.cc:260] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 7.31, Last appended by leader: 31, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.967516 9422 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.968384 9422 raft_consensus.cc:2396] T 2a5f4a0cb82e41878cbb63d99010c0a6 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 8 because replica has last-logged OpId of term: 7 index: 31, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 30.
I20240502 01:59:33.967110 9156 raft_consensus.cc:1226] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Rejecting Update request from peer 8d72715586b447d69353ef2fdab02ae4 for earlier term 7. Current term is 8. Ops: [7.31-7.31]
I20240502 01:59:33.969648 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5f4a0cb82e41878cbb63d99010c0a6" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 8 candidate_status { last_received { term: 7 index: 30 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:33.969763 9290 raft_consensus.cc:3037] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.970594 9290 raft_consensus.cc:2454] T 2a5f4a0cb82e41878cbb63d99010c0a6 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 in term 8.
I20240502 01:59:33.971547 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3c859176746d697e47b54b2d5b9a2" candidate_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" candidate_term: 7 candidate_status { last_received { term: 6 index: 25 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:33.971755 9425 raft_consensus.cc:2396] T 3ec3c859176746d697e47b54b2d5b9a2 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efc6b28dd0a34e6f8bb7f4020d2dc6a7 for term 7 because replica has last-logged OpId of term: 6 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 25.
I20240502 01:59:33.972432 9089 leader_election.cc:302] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:33.972750 9089 leader_election.cc:302] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters: 8d72715586b447d69353ef2fdab02ae4
I20240502 01:59:33.972827 11540 raft_consensus.cc:2726] T 3ec3c859176746d697e47b54b2d5b9a2 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 01:59:33.972947 11270 raft_consensus.cc:2781] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:33.973013 11270 raft_consensus.cc:686] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Leader. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:33.973115 11270 consensus_queue.cc:237] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 7.30, Last appended by leader: 30, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:33.974670 8872 catalog_manager.cc:5484] T 2a5f4a0cb82e41878cbb63d99010c0a6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194). New cstate: current_term: 8 leader_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
W20240502 01:59:33.987605 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60772: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:33.993908 9020 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 23. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
I20240502 01:59:33.995640 11270 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20240502 01:59:33.997712 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:33.997822 9020 raft_consensus.cc:482] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:33.997874 9020 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:33.998977 9020 raft_consensus.cc:504] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:33.999323 9020 leader_election.cc:288] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:33.999881 9290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.000008 9290 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.000934 9290 raft_consensus.cc:2454] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 8.
I20240502 01:59:34.001307 8956 leader_election.cc:302] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:34.001469 11483 raft_consensus.cc:2781] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:34.001549 11483 raft_consensus.cc:686] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 8 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.001662 11483 consensus_queue.cc:237] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 7.24, Last appended by leader: 24, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.003074 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c11fb113b44a9eb9c2370d7c88760b" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 7 index: 24 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.004321 9290 raft_consensus.cc:1261] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Refusing update from remote peer efc6b28dd0a34e6f8bb7f4020d2dc6a7: Log matching property violated. Preceding OpId in replica: term: 7 index: 23. Preceding OpId from leader: term: 8 index: 25. (index mismatch)
I20240502 01:59:34.004446 9020 raft_consensus.cc:1226] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb [term 8 LEADER]: Rejecting Update request from peer 8d72715586b447d69353ef2fdab02ae4 for earlier term 7. Current term is 8. Ops: [7.25-7.25]
I20240502 01:59:34.005010 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.005113 9020 raft_consensus.cc:482] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.005160 9020 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.005095 10585 consensus_queue.cc:1046] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 }, Status: INVALID_TERM, Last received: 7.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20240502 01:59:34.005218 10585 consensus_queue.cc:1225] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [LEADER]: Peer d6bd594b464a44f89a636ca1b758aefb log is divergent from this leader: its last log entry 8.25 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 24
I20240502 01:59:34.005224 11539 consensus_queue.cc:1035] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20240502 01:59:34.005275 10585 raft_consensus.cc:3032] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:34.005321 10585 raft_consensus.cc:729] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:34.005431 10585 consensus_queue.cc:260] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 7.25, Last appended by leader: 25, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.005545 10585 raft_consensus.cc:3037] T 74c11fb113b44a9eb9c2370d7c88760b P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.005973 9020 raft_consensus.cc:504] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.006304 9020 leader_election.cc:288] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:34.006635 9290 raft_consensus.cc:1226] T 74c11fb113b44a9eb9c2370d7c88760b P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Rejecting Update request from peer 8d72715586b447d69353ef2fdab02ae4 for earlier term 7. Current term is 8. Ops: [7.25-7.25]
I20240502 01:59:34.006844 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.006953 9158 raft_consensus.cc:3032] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.006992 9158 raft_consensus.cc:729] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.007105 9158 consensus_queue.cc:260] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.14, Last appended by leader: 14, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.007259 9158 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.007421 8875 catalog_manager.cc:5484] T 74c11fb113b44a9eb9c2370d7c88760b P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 8 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:34.008131 9158 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.008517 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06bfc16dee5546a0a33fe56c32a4bd02" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.008610 9294 raft_consensus.cc:3037] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.009474 9294 raft_consensus.cc:2454] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.009924 8956 leader_election.cc:302] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:34.010336 11483 raft_consensus.cc:2781] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election won for term 9
I20240502 01:59:34.010432 11483 raft_consensus.cc:686] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [term 9 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.010537 11483 consensus_queue.cc:237] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.14, Last appended by leader: 14, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.011341 11529 mvcc.cc:205] Tried to move back new op lower bound from 7023063752675426304 to 7023063752522596352. Current Snapshot: MvccSnapshot[applied={T|T < 7023063752675426304}]
I20240502 01:59:34.012277 9023 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.012391 9023 raft_consensus.cc:482] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.012444 9023 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.013357 9023 raft_consensus.cc:504] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.013669 9023 leader_election.cc:288] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:34.014344 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 7 index: 15 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.014453 9422 raft_consensus.cc:3032] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:34.014499 9422 raft_consensus.cc:729] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:34.014621 9422 consensus_queue.cc:260] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 7.15, Last appended by leader: 15, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.014781 9422 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.015667 9422 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 8.
I20240502 01:59:34.016191 8956 leader_election.cc:302] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:34.016320 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf237c991ee4754817d5de7d34dc7ab" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 7 index: 15 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.016415 9291 raft_consensus.cc:3037] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
W20240502 01:59:34.016494 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:34.016635 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "196e584cf0b74034a99def7a33872965"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.016743 9020 raft_consensus.cc:482] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.016804 9020 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.016790 11483 raft_consensus.cc:2781] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:34.017030 11483 raft_consensus.cc:686] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [term 8 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.017138 11483 consensus_queue.cc:237] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 7.15, Last appended by leader: 15, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.017251 9291 raft_consensus.cc:2454] T 5cf237c991ee4754817d5de7d34dc7ab P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 8.
I20240502 01:59:34.017951 8872 catalog_manager.cc:5484] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 8 to 9, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 9 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:34.018644 9020 raft_consensus.cc:504] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.019119 9020 leader_election.cc:288] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:34.019763 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.019867 9158 raft_consensus.cc:3032] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.019905 9158 raft_consensus.cc:729] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.020020 9158 consensus_queue.cc:260] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.14, Last appended by leader: 14, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.020169 9158 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.021066 9158 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.021620 8957 leader_election.cc:302] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:34.021782 11352 raft_consensus.cc:2781] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election won for term 9
W20240502 01:59:34.021813 9253 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60780: Illegal state: replica ee7bf4c5836f4264b281adc40ecdac9d is not leader of this config: current role FOLLOWER
I20240502 01:59:34.021848 11352 raft_consensus.cc:686] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [term 9 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.021950 11352 consensus_queue.cc:237] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.14, Last appended by leader: 14, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.022868 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "196e584cf0b74034a99def7a33872965" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.022888 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.022970 9425 raft_consensus.cc:3037] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.022986 9020 raft_consensus.cc:482] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.023034 9020 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.023375 8878 catalog_manager.cc:5484] T 5cf237c991ee4754817d5de7d34dc7ab P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 7 to 8, leader changed from 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 8 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } }
I20240502 01:59:34.023897 9425 raft_consensus.cc:2454] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.023900 9020 raft_consensus.cc:504] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.024210 9020 leader_election.cc:288] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:34.024502 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.024549 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fa0c3c70ef430fad68cdf679f9c2b6" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.024597 9291 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.024626 9158 raft_consensus.cc:3032] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:34.024668 9158 raft_consensus.cc:729] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.024781 9158 consensus_queue.cc:260] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.024906 9158 raft_consensus.cc:3037] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.025432 9291 raft_consensus.cc:2454] T 30fa0c3c70ef430fad68cdf679f9c2b6 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 8.
I20240502 01:59:34.025697 9158 raft_consensus.cc:2396] T 30fa0c3c70ef430fad68cdf679f9c2b6 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 01:59:34.026980 8956 leader_election.cc:302] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters: efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:34.027215 11483 raft_consensus.cc:2781] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:34.027309 11483 raft_consensus.cc:686] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [term 8 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.027324 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.027426 11483 consensus_queue.cc:237] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.027417 9020 raft_consensus.cc:482] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.027992 8878 catalog_manager.cc:5484] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 8 to 9, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 9 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } }
I20240502 01:59:34.028137 9020 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.028839 9020 raft_consensus.cc:504] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.029107 9291 raft_consensus.cc:1261] T 06bfc16dee5546a0a33fe56c32a4bd02 P ee7bf4c5836f4264b281adc40ecdac9d [term 9 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 8 index: 14. Preceding OpId from leader: term: 9 index: 16. (index mismatch)
I20240502 01:59:34.029155 9158 raft_consensus.cc:1261] T 06bfc16dee5546a0a33fe56c32a4bd02 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 8 index: 14. Preceding OpId from leader: term: 9 index: 16. (index mismatch)
I20240502 01:59:34.029428 9020 leader_election.cc:288] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:34.029850 11483 consensus_queue.cc:1035] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:34.030458 11483 consensus_queue.cc:1035] T 06bfc16dee5546a0a33fe56c32a4bd02 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:34.031152 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.031294 9158 raft_consensus.cc:3032] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.031380 9158 raft_consensus.cc:729] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.031574 9158 consensus_queue.cc:260] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 8.21, Last appended by leader: 21, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.032617 8878 catalog_manager.cc:5484] T 30fa0c3c70ef430fad68cdf679f9c2b6 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 7 to 8, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 8 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:34.032745 9158 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.032992 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "133e3b04d21a4223b80dfeae975ca78f" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 7 index: 20 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.033207 9291 raft_consensus.cc:3037] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.033800 11542 mvcc.cc:205] Tried to move back new op lower bound from 7023063752819867648 to 7023063752748216320. Current Snapshot: MvccSnapshot[applied={T|T < 7023063752819867648 or (T in {7023063752819867648})}]
I20240502 01:59:34.034119 9158 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 8 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 20.
I20240502 01:59:34.034703 9291 raft_consensus.cc:2396] T 133e3b04d21a4223b80dfeae975ca78f P ee7bf4c5836f4264b281adc40ecdac9d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 7 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 20.
I20240502 01:59:34.036476 8957 leader_election.cc:302] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:34.036736 11483 raft_consensus.cc:2726] T 133e3b04d21a4223b80dfeae975ca78f P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 01:59:34.040555 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4f33c787846045d08c1ad152632509e7"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.040670 9020 raft_consensus.cc:482] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.040726 9020 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.041483 9020 raft_consensus.cc:504] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.041781 9020 leader_election.cc:288] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:34.043321 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.043426 9425 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.043704 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f33c787846045d08c1ad152632509e7" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 5 index: 13 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.043800 9291 raft_consensus.cc:3037] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.044257 9425 raft_consensus.cc:2454] T 4f33c787846045d08c1ad152632509e7 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 8.
I20240502 01:59:34.044829 8956 leader_election.cc:302] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:34.044976 11264 raft_consensus.cc:2781] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election won for term 8
I20240502 01:59:34.045055 11264 raft_consensus.cc:686] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [term 8 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.045181 11264 consensus_queue.cc:237] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 5.13, Last appended by leader: 13, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.045044 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.045720 9020 raft_consensus.cc:482] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.045773 9020 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.046538 9291 raft_consensus.cc:2396] T 4f33c787846045d08c1ad152632509e7 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 8 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 13.
I20240502 01:59:34.046691 8875 catalog_manager.cc:5484] T 4f33c787846045d08c1ad152632509e7 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 6 to 8, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 8 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:34.047467 9020 raft_consensus.cc:504] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.048049 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.048142 9156 raft_consensus.cc:3032] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.048187 9156 raft_consensus.cc:729] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.048313 9156 consensus_queue.cc:260] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 8.18, Last appended by leader: 18, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.048439 9156 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.048841 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44fa89e0dff6462d8782bff0e9f06165" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.048945 9294 raft_consensus.cc:3037] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.049355 9156 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 8 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:34.049517 9020 leader_election.cc:288] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:34.049840 9294 raft_consensus.cc:2396] T 44fa89e0dff6462d8782bff0e9f06165 P ee7bf4c5836f4264b281adc40ecdac9d [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 7 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 01:59:34.051398 8956 leader_election.cc:302] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: ee7bf4c5836f4264b281adc40ecdac9d, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:34.052127 11483 raft_consensus.cc:2726] T 44fa89e0dff6462d8782bff0e9f06165 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 01:59:34.052809 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.052944 9020 raft_consensus.cc:482] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.052996 9020 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.054905 9020 raft_consensus.cc:504] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.055351 9020 leader_election.cc:288] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:34.055437 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.055528 9156 raft_consensus.cc:3032] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.055562 9156 raft_consensus.cc:729] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.055672 9156 consensus_queue.cc:260] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } }
I20240502 01:59:34.055789 9156 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.056550 9422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a305710d4443bbac2a13f4ea8d5224" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.056645 9156 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20240502 01:59:34.056670 9422 raft_consensus.cc:3037] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.057595 9422 raft_consensus.cc:2396] T 82a305710d4443bbac2a13f4ea8d5224 P 8d72715586b447d69353ef2fdab02ae4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20240502 01:59:34.058012 8956 leader_election.cc:302] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: 8d72715586b447d69353ef2fdab02ae4, efc6b28dd0a34e6f8bb7f4020d2dc6a7
I20240502 01:59:34.058945 11264 raft_consensus.cc:2726] T 82a305710d4443bbac2a13f4ea8d5224 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 01:59:34.058725 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.060233 9020 raft_consensus.cc:482] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.060292 9020 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:34.061127 9020 raft_consensus.cc:504] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.061712 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 7 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.061811 9425 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:34.062741 9425 raft_consensus.cc:2454] T f2f2efeea25947f4bab37ef8a67c6d01 P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 7.
I20240502 01:59:34.063031 8956 leader_election.cc:302] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:34.063172 11264 raft_consensus.cc:2781] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Leader election won for term 7
I20240502 01:59:34.063268 11264 raft_consensus.cc:686] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [term 7 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.063371 11264 consensus_queue.cc:237] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 4.14, Last appended by leader: 14, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.064452 9020 leader_election.cc:288] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 7 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
W20240502 01:59:34.064702 9118 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43494: Illegal state: replica efc6b28dd0a34e6f8bb7f4020d2dc6a7 is not leader of this config: current role FOLLOWER
I20240502 01:59:34.064873 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f2efeea25947f4bab37ef8a67c6d01" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 7 candidate_status { last_received { term: 4 index: 14 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.065218 9291 raft_consensus.cc:3037] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 6 FOLLOWER]: Advancing to term 7
I20240502 01:59:34.066098 9291 raft_consensus.cc:2396] T f2f2efeea25947f4bab37ef8a67c6d01 P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 7 because replica has last-logged OpId of term: 5 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 14.
I20240502 01:59:34.066009 8871 catalog_manager.cc:5484] T f2f2efeea25947f4bab37ef8a67c6d01 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 5 to 7, leader changed from ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 7 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:34.066490 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.066599 9020 raft_consensus.cc:482] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.066659 9020 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.067425 9020 raft_consensus.cc:504] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.067952 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.067997 9020 leader_election.cc:288] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:34.068043 9425 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.068509 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.068531 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82c9e2ba9224bad99c733a90acbfe3e" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.068609 9020 raft_consensus.cc:482] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.068619 9156 raft_consensus.cc:3032] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.068655 9156 raft_consensus.cc:729] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.068660 9020 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.068792 9156 consensus_queue.cc:260] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.11, Last appended by leader: 11, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.068919 9156 raft_consensus.cc:3037] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.069008 9425 raft_consensus.cc:2454] T d82c9e2ba9224bad99c733a90acbfe3e P 8d72715586b447d69353ef2fdab02ae4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.069298 8956 leader_election.cc:302] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d72715586b447d69353ef2fdab02ae4, d6bd594b464a44f89a636ca1b758aefb; no voters:
I20240502 01:59:34.069829 11483 raft_consensus.cc:2781] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election won for term 9
I20240502 01:59:34.069907 11483 raft_consensus.cc:686] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [term 9 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
W20240502 01:59:34.070050 9384 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34146: Illegal state: replica 8d72715586b447d69353ef2fdab02ae4 is not leader of this config: current role FOLLOWER
I20240502 01:59:34.070017 11483 consensus_queue.cc:237] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.070452 9020 raft_consensus.cc:504] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.070966 9156 raft_consensus.cc:2396] T d82c9e2ba9224bad99c733a90acbfe3e P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 9 because replica has last-logged OpId of term: 8 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 10.
I20240502 01:59:34.071364 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 18 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.071406 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c02e74f8b37e4397aa5dc4a3e9acbe59" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 18 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.071455 9291 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.071498 9158 raft_consensus.cc:3032] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.071533 9158 raft_consensus.cc:729] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.071672 9158 consensus_queue.cc:260] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.18, Last appended by leader: 18, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.071863 9158 raft_consensus.cc:3037] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.072153 8875 catalog_manager.cc:5484] T d82c9e2ba9224bad99c733a90acbfe3e P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 8 to 9, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 9 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } }
I20240502 01:59:34.072324 9291 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P ee7bf4c5836f4264b281adc40ecdac9d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.072566 8956 leader_election.cc:302] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, ee7bf4c5836f4264b281adc40ecdac9d; no voters:
I20240502 01:59:34.072724 11352 raft_consensus.cc:2781] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election won for term 9
I20240502 01:59:34.072805 11352 raft_consensus.cc:686] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [term 9 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.072880 9020 leader_election.cc:288] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:34.072911 11352 consensus_queue.cc:237] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.18, Last appended by leader: 18, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.073519 9158 raft_consensus.cc:2454] T c02e74f8b37e4397aa5dc4a3e9acbe59 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.074548 9425 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P 8d72715586b447d69353ef2fdab02ae4 [term 9 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 8 index: 14. Preceding OpId from leader: term: 9 index: 16. (index mismatch)
I20240502 01:59:34.075075 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.075186 9020 raft_consensus.cc:482] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.075245 9020 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.075295 9158 raft_consensus.cc:1261] T 196e584cf0b74034a99def7a33872965 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Refusing update from remote peer d6bd594b464a44f89a636ca1b758aefb: Log matching property violated. Preceding OpId in replica: term: 8 index: 14. Preceding OpId from leader: term: 9 index: 16. (index mismatch)
I20240502 01:59:34.075376 11565 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:34.075744 11483 consensus_queue.cc:1035] T 196e584cf0b74034a99def7a33872965 P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 01:59:34.075953 8875 catalog_manager.cc:5484] T c02e74f8b37e4397aa5dc4a3e9acbe59 P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 8 to 9, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 9 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:34.076102 9020 raft_consensus.cc:504] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.076421 9020 leader_election.cc:288] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031), ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679)
I20240502 01:59:34.077039 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 25 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.077152 9153 raft_consensus.cc:3032] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.077193 9153 raft_consensus.cc:729] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.077304 9153 consensus_queue.cc:260] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 8.25, Last appended by leader: 25, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.077450 9153 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.078322 9153 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.080794 8957 leader_election.cc:302] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb, efc6b28dd0a34e6f8bb7f4020d2dc6a7; no voters:
I20240502 01:59:34.080955 11565 raft_consensus.cc:2781] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Leader election won for term 9
I20240502 01:59:34.081071 11565 raft_consensus.cc:686] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [term 9 LEADER]: Becoming Leader. State: Replica: d6bd594b464a44f89a636ca1b758aefb, State: Running, Role: LEADER
I20240502 01:59:34.081180 11565 consensus_queue.cc:237] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 8.25, Last appended by leader: 25, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } }
I20240502 01:59:34.081729 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecbaecc48414a56a09af563e081362c" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 8 index: 25 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.081837 9291 raft_consensus.cc:3037] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.082620 9291 raft_consensus.cc:2454] T 7ecbaecc48414a56a09af563e081362c P ee7bf4c5836f4264b281adc40ecdac9d [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6bd594b464a44f89a636ca1b758aefb in term 9.
I20240502 01:59:34.084968 8875 catalog_manager.cc:5484] T 7ecbaecc48414a56a09af563e081362c P d6bd594b464a44f89a636ca1b758aefb reported cstate change: term changed from 8 to 9, leader changed from efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194) to d6bd594b464a44f89a636ca1b758aefb (127.19.43.193). New cstate: current_term: 9 leader_uuid: "d6bd594b464a44f89a636ca1b758aefb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } }
I20240502 01:59:34.087360 10895 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 01:59:34.087633 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.087759 9020 raft_consensus.cc:482] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.087807 9020 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.088771 9020 raft_consensus.cc:504] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.089049 9020 leader_election.cc:288] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), 8d72715586b447d69353ef2fdab02ae4 (127.19.43.196:32843)
I20240502 01:59:34.089565 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 5 index: 37 } } ignore_live_leader: true dest_uuid: "ee7bf4c5836f4264b281adc40ecdac9d"
I20240502 01:59:34.089668 9294 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.090312 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f5ad44d23f4c6e802467fdc01cb8be" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 8 candidate_status { last_received { term: 5 index: 37 } } ignore_live_leader: true dest_uuid: "8d72715586b447d69353ef2fdab02ae4"
I20240502 01:59:34.090411 9425 raft_consensus.cc:3032] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 01:59:34.090451 9425 raft_consensus.cc:729] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8d72715586b447d69353ef2fdab02ae4, State: Running, Role: LEADER
I20240502 01:59:34.090631 9294 raft_consensus.cc:2396] T 40f5ad44d23f4c6e802467fdc01cb8be P ee7bf4c5836f4264b281adc40ecdac9d [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 8 because replica has last-logged OpId of term: 6 index: 39, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 37.
I20240502 01:59:34.090579 9425 consensus_queue.cc:260] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 7.40, Last appended by leader: 40, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "8d72715586b447d69353ef2fdab02ae4" member_type: VOTER last_known_addr { host: "127.19.43.196" port: 32843 } } peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } }
I20240502 01:59:34.090713 9425 raft_consensus.cc:3037] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 7 FOLLOWER]: Advancing to term 8
I20240502 01:59:34.091567 9425 raft_consensus.cc:2396] T 40f5ad44d23f4c6e802467fdc01cb8be P 8d72715586b447d69353ef2fdab02ae4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d6bd594b464a44f89a636ca1b758aefb for term 8 because replica has last-logged OpId of term: 7 index: 40, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 37.
I20240502 01:59:34.091904 8956 leader_election.cc:302] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6bd594b464a44f89a636ca1b758aefb; no voters: 8d72715586b447d69353ef2fdab02ae4, ee7bf4c5836f4264b281adc40ecdac9d
I20240502 01:59:34.092422 11352 raft_consensus.cc:2726] T 40f5ad44d23f4c6e802467fdc01cb8be P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 01:59:34.097816 9020 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573"
dest_uuid: "d6bd594b464a44f89a636ca1b758aefb"
from {username='slave'} at 127.0.0.1:56752
I20240502 01:59:34.098304 9020 raft_consensus.cc:482] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 01:59:34.098425 9020 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.099323 9020 raft_consensus.cc:504] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.099714 9020 leader_election.cc:288] T 0f04b862773348a78aa4596b0f42d573 P d6bd594b464a44f89a636ca1b758aefb [CANDIDATE]: Term 9 election: Requested vote from peers ee7bf4c5836f4264b281adc40ecdac9d (127.19.43.195:46679), efc6b28dd0a34e6f8bb7f4020d2dc6a7 (127.19.43.194:35031)
I20240502 01:59:34.099974 9156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb" candidate_term: 9 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7"
I20240502 01:59:34.100380 9156 raft_consensus.cc:3032] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 01:59:34.100432 9156 raft_consensus.cc:729] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efc6b28dd0a34e6f8bb7f4020d2dc6a7, State: Running, Role: LEADER
I20240502 01:59:34.100555 9156 consensus_queue.cc:260] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6bd594b464a44f89a636ca1b758aefb" member_type: VOTER last_known_addr { host: "127.19.43.193" port: 36545 } } peers { permanent_uuid: "ee7bf4c5836f4264b281adc40ecdac9d" member_type: VOTER last_known_addr { host: "127.19.43.195" port: 46679 } } peers { permanent_uuid: "efc6b28dd0a34e6f8bb7f4020d2dc6a7" member_type: VOTER last_known_addr { host: "127.19.43.194" port: 35031 } }
I20240502 01:59:34.100679 9156 raft_consensus.cc:3037] T 0f04b862773348a78aa4596b0f42d573 P efc6b28dd0a34e6f8bb7f4020d2dc6a7 [term 8 FOLLOWER]: Advancing to term 9
I20240502 01:59:34.101187 9291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f04b862773348a78aa4596b0f42d573" candidate_uuid: "d6bd594b464a44f89a636ca1b758aefb"